Max level shown:
1 [ Nov 13 07:58:42 Enabled. ]
2 [ Nov 13 07:58:42 Rereading configuration. ]
3 [ Nov 13 07:58:43 Rereading configuration. ]
4 [ Nov 13 07:58:47 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/oximeter-collector/bin/oximeter run /var/svc/manifest/site/oximeter/config.toml --address [fd00:1122:3344:101::d]:12223 --id 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d &"). ]
5 [ Nov 13 07:58:47 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72025-11-13T07:58:48.724ZDEBGoximeter: registered DTrace probes
82025-11-13T07:58:48.748ZINFOoximeter: starting oximeter server file = oximeter/collector/src/lib.rs:264
92025-11-13T07:58:48.751ZDEBGoximeter: creating ClickHouse client
102025-11-13T07:58:48.906ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
112025-11-13T07:58:59.805ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:900 id = d44d737b-0153-48a6-bcaf-41cbb7231e15
122025-11-13T07:58:59.805ZDEBGoximeter (oximeter-agent): oximeter database does not exist, creating collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
132025-11-13T07:58:59.806ZINFOoximeter (clickhouse-client): reading db version collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:800 id = d44d737b-0153-48a6-bcaf-41cbb7231e15
142025-11-13T07:58:59.807ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:900 id = d44d737b-0153-48a6-bcaf-41cbb7231e15
152025-11-13T07:58:59.807ZINFOoximeter (clickhouse-client): read oximeter database version collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:805 id = d44d737b-0153-48a6-bcaf-41cbb7231e15 version = 0
162025-11-13T07:58:59.807ZINFOoximeter (clickhouse-client): initializing oximeter schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:810 id = d44d737b-0153-48a6-bcaf-41cbb7231e15
172025-11-13T07:58:59.807ZDEBGoximeter (clickhouse-client): initializing ClickHouse database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
182025-11-13T07:59:00.182ZINFOoximeter (clickhouse-client): inserting current version collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:831 id = d44d737b-0153-48a6-bcaf-41cbb7231e15 version = 14
192025-11-13T07:59:00.219ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = 88ee250b-22e7-48d2-90b3-c9938e0201bb
202025-11-13T07:59:00.248ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /info versions = all
212025-11-13T07:59:00.248ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers versions = all
222025-11-13T07:59:00.248ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = DELETE path = /producers/{producer_id} versions = all
232025-11-13T07:59:00.248ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers/{producer_id} versions = all
242025-11-13T07:59:00.249ZINFOoximeter (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::d]:12223
252025-11-13T07:59:00.249ZDEBGoximeter (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::d]:12223
262025-11-13T07:59:00.249ZDEBGoximeter: registered USDT probes
272025-11-13T07:59:00.249ZDEBGoximeter: contacting nexus
282025-11-13T07:59:00.401ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors
292025-11-13T07:59:00.469ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors", status: 204, headers: {"x-request-id": "7540ccb3-3e6d-4add-8327-e27174fb5bcb", "date": "Thu, 13 Nov 2025 07:59:00 GMT"} })
302025-11-13T07:59:00.469ZINFOoximeter: oximeter registered with nexus file = oximeter/collector/src/lib.rs:427 id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d
312025-11-13T07:59:00.472ZINFOoximeter (oximeter-agent): starting refresh list task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:554
322025-11-13T07:59:00.473ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
332025-11-13T07:59:00.476ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
342025-11-13T07:59:00.522ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c5e8038-03d6-475b-8ae7-2e7c9f8b015b", "content-length": "29", "date": "Thu, 13 Nov 2025 07:59:00 GMT"} })
352025-11-13T07:59:00.522ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 0 n_pruned_tasks = 0
362025-11-13T07:59:15.472ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
372025-11-13T07:59:15.473ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
382025-11-13T07:59:15.475ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "c88e4504-e2b8-4882-acc5-c43497b1a6a4", "content-length": "572", "date": "Thu, 13 Nov 2025 07:59:15 GMT"} })
392025-11-13T07:59:15.482ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4OTE2ZWY3ZS1iOWU4LTRjNDktODVkNy02NjYyNTFhYWI4MWQifX0%3D
402025-11-13T07:59:15.484ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI4OTE2ZWY3ZS1iOWU4LTRjNDktODVkNy02NjYyNTFhYWI4MWQifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "521f2879-c156-457b-a295-d39f3c0b56b0", "content-length": "29", "date": "Thu, 13 Nov 2025 07:59:15 GMT"} })
412025-11-13T07:59:15.484ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:8001 collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f
422025-11-13T07:59:15.484ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::b]:62328 collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435
432025-11-13T07:59:15.484ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:8001 collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d
442025-11-13T07:59:15.484ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 3 n_pruned_tasks = 0
452025-11-13T07:59:15.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
462025-11-13T07:59:15.485ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
472025-11-13T07:59:15.485ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
482025-11-13T07:59:15.485ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
492025-11-13T07:59:15.517ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
502025-11-13T07:59:15.517ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
512025-11-13T07:59:15.581ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
522025-11-13T07:59:15.588ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 5 producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
532025-11-13T07:59:15.650ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
542025-11-13T07:59:15.652ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
552025-11-13T07:59:15.656ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
562025-11-13T07:59:15.662ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
572025-11-13T07:59:16.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
582025-11-13T07:59:16.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
592025-11-13T07:59:16.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
602025-11-13T07:59:16.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
612025-11-13T07:59:16.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
622025-11-13T07:59:16.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
632025-11-13T07:59:17.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
642025-11-13T07:59:17.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
652025-11-13T07:59:17.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
662025-11-13T07:59:17.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
672025-11-13T07:59:17.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
682025-11-13T07:59:17.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
692025-11-13T07:59:18.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
702025-11-13T07:59:18.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
712025-11-13T07:59:18.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
722025-11-13T07:59:18.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
732025-11-13T07:59:18.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
742025-11-13T07:59:18.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
752025-11-13T07:59:19.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
762025-11-13T07:59:19.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
772025-11-13T07:59:19.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
782025-11-13T07:59:19.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
792025-11-13T07:59:19.486ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
802025-11-13T07:59:19.486ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
812025-11-13T07:59:20.219ZDEBGoximeter (results-sink): inserting 215 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
822025-11-13T07:59:20.219ZDEBGoximeter (results-sink-cluster): inserting 215 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
832025-11-13T07:59:20.219ZDEBGoximeter (clickhouse-client): unrolling 215 total samples collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
842025-11-13T07:59:20.219ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
852025-11-13T07:59:20.221ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:collections"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "base_route", field_type: String, source: Metric, description: "" }, FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "producer_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "producer_ip", field_type: IpAddr, source: Metric, description: "" }, FieldSchema { name: "producer_port", field_type: U16, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.219807423Z } timeseries_name = oximeter_collector:collections
862025-11-13T07:59:20.229ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
872025-11-13T07:59:20.236ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.229151944Z } timeseries_name = http_service:request_latency_histogram
882025-11-13T07:59:20.237ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
892025-11-13T07:59:20.240ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:cpus_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.237452430Z } timeseries_name = collection_target:cpus_provisioned
902025-11-13T07:59:20.240ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
912025-11-13T07:59:20.241ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:ram_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.240446149Z } timeseries_name = collection_target:ram_provisioned
922025-11-13T07:59:20.241ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
932025-11-13T07:59:20.242ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:virtual_disk_space_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.241338462Z } timeseries_name = collection_target:virtual_disk_space_provisioned
942025-11-13T07:59:20.242ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
952025-11-13T07:59:20.243ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.242388007Z } timeseries_name = ddm_router:originated_underlay_prefixes
962025-11-13T07:59:20.243ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
972025-11-13T07:59:20.244ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.243278285Z } timeseries_name = ddm_router:originated_tunnel_endpoints
982025-11-13T07:59:20.244ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
992025-11-13T07:59:20.245ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.244217863Z } timeseries_name = ddm_session:solicitations_sent
1002025-11-13T07:59:20.245ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1012025-11-13T07:59:20.245ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.245121631Z } timeseries_name = ddm_session:solicitations_received
1022025-11-13T07:59:20.246ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1032025-11-13T07:59:20.246ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.246027569Z } timeseries_name = ddm_session:advertisements_sent
1042025-11-13T07:59:20.246ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1052025-11-13T07:59:20.247ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.246942717Z } timeseries_name = ddm_session:advertisements_received
1062025-11-13T07:59:20.247ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1072025-11-13T07:59:20.248ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_expirations"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.247800098Z } timeseries_name = ddm_session:peer_expirations
1082025-11-13T07:59:20.248ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1092025-11-13T07:59:20.249ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_address_changes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.248786143Z } timeseries_name = ddm_session:peer_address_changes
1102025-11-13T07:59:20.249ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1112025-11-13T07:59:20.250ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_sessions_established"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.249708741Z } timeseries_name = ddm_session:peer_sessions_established
1122025-11-13T07:59:20.250ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1132025-11-13T07:59:20.251ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.250625929Z } timeseries_name = ddm_session:updates_sent
1142025-11-13T07:59:20.251ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1152025-11-13T07:59:20.252ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.251575758Z } timeseries_name = ddm_session:updates_received
1162025-11-13T07:59:20.252ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1172025-11-13T07:59:20.253ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:update_send_fail"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.252544447Z } timeseries_name = ddm_session:update_send_fail
1182025-11-13T07:59:20.253ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1192025-11-13T07:59:20.254ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.253460468Z } timeseries_name = ddm_session:imported_underlay_prefixes
1202025-11-13T07:59:20.254ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1212025-11-13T07:59:20.255ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:20.254409514Z } timeseries_name = ddm_session:imported_tunnel_endpoints
1222025-11-13T07:59:20.261ZDEBGoximeter (clickhouse-client): inserting 19 new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1232025-11-13T07:59:20.270ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 6 table_name = fields_ipaddr
1242025-11-13T07:59:20.272ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 67 table_name = fields_string
1252025-11-13T07:59:20.274ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 12 table_name = fields_u16
1262025-11-13T07:59:20.275ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 74 table_name = fields_uuid
1272025-11-13T07:59:20.279ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 153 table_name = measurements_cumulativeu64
1282025-11-13T07:59:20.284ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 6 table_name = measurements_histogramu64
1292025-11-13T07:59:20.286ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 6 table_name = measurements_i64
1302025-11-13T07:59:20.287ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 50 table_name = measurements_u64
1312025-11-13T07:59:20.322ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
1322025-11-13T07:59:20.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1332025-11-13T07:59:20.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1342025-11-13T07:59:20.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1352025-11-13T07:59:20.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1362025-11-13T07:59:20.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1372025-11-13T07:59:20.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1382025-11-13T07:59:21.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1392025-11-13T07:59:21.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1402025-11-13T07:59:21.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1412025-11-13T07:59:21.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1422025-11-13T07:59:21.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1432025-11-13T07:59:21.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1442025-11-13T07:59:22.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1452025-11-13T07:59:22.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1462025-11-13T07:59:22.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1472025-11-13T07:59:22.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1482025-11-13T07:59:22.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1492025-11-13T07:59:22.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1502025-11-13T07:59:23.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1512025-11-13T07:59:23.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1522025-11-13T07:59:23.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1532025-11-13T07:59:23.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1542025-11-13T07:59:23.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1552025-11-13T07:59:23.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1562025-11-13T07:59:24.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1572025-11-13T07:59:24.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1582025-11-13T07:59:24.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1592025-11-13T07:59:24.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1602025-11-13T07:59:24.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1612025-11-13T07:59:24.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1622025-11-13T07:59:25.219ZDEBGoximeter (results-sink): inserting 200 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
1632025-11-13T07:59:25.219ZDEBGoximeter (results-sink-cluster): inserting 200 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
1642025-11-13T07:59:25.220ZDEBGoximeter (clickhouse-client): unrolling 200 total samples collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
1652025-11-13T07:59:25.229ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 52 table_name = fields_string
1662025-11-13T07:59:25.231ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 56 table_name = fields_uuid
1672025-11-13T07:59:25.233ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 150 table_name = measurements_cumulativeu64
1682025-11-13T07:59:25.235ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 50 table_name = measurements_u64
1692025-11-13T07:59:25.320ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
1702025-11-13T07:59:25.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
1712025-11-13T07:59:25.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
1722025-11-13T07:59:25.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1732025-11-13T07:59:25.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1742025-11-13T07:59:25.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1752025-11-13T07:59:25.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1762025-11-13T07:59:25.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1772025-11-13T07:59:25.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1782025-11-13T07:59:25.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 5 producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
1792025-11-13T07:59:26.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1802025-11-13T07:59:26.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1812025-11-13T07:59:26.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1822025-11-13T07:59:26.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1832025-11-13T07:59:26.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1842025-11-13T07:59:26.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1852025-11-13T07:59:27.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1862025-11-13T07:59:27.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1872025-11-13T07:59:27.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1882025-11-13T07:59:27.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1892025-11-13T07:59:27.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1902025-11-13T07:59:27.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1912025-11-13T07:59:28.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1922025-11-13T07:59:28.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1932025-11-13T07:59:28.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1942025-11-13T07:59:28.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1952025-11-13T07:59:28.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1962025-11-13T07:59:28.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1972025-11-13T07:59:29.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1982025-11-13T07:59:29.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1992025-11-13T07:59:29.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2002025-11-13T07:59:29.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2012025-11-13T07:59:29.486ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2022025-11-13T07:59:29.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2032025-11-13T07:59:30.219ZDEBGoximeter (results-sink): inserting 229 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
2042025-11-13T07:59:30.219ZDEBGoximeter (results-sink-cluster): inserting 229 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
2052025-11-13T07:59:30.219ZDEBGoximeter (clickhouse-client): unrolling 229 total samples collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
2062025-11-13T07:59:30.230ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 92 table_name = fields_string
2072025-11-13T07:59:30.232ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 20 table_name = fields_u16
2082025-11-13T07:59:30.234ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 81 table_name = fields_uuid
2092025-11-13T07:59:30.235ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 150 table_name = measurements_cumulativeu64
2102025-11-13T07:59:30.238ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 20 table_name = measurements_histogramu64
2112025-11-13T07:59:30.240ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 9 table_name = measurements_i64
2122025-11-13T07:59:30.242ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 50 table_name = measurements_u64
2132025-11-13T07:59:30.320ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
2142025-11-13T07:59:30.472ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
2152025-11-13T07:59:30.472ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
2162025-11-13T07:59:30.476ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "49bf49a6-5542-4574-98b1-ce4f003bdfc8", "content-length": "710", "date": "Thu, 13 Nov 2025 07:59:30 GMT"} })
2172025-11-13T07:59:30.476ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiZTkwZTUxNi04MjRlLTRmZmEtOGIzNC05N2E3OTRmZDcxMTgifX0%3D
2182025-11-13T07:59:30.478ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiZTkwZTUxNi04MjRlLTRmZmEtOGIzNC05N2E3OTRmZDcxMTgifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "8829c8e6-74c3-426e-9d2e-b961803afb5f", "content-length": "29", "date": "Thu, 13 Nov 2025 07:59:30 GMT"} })
2192025-11-13T07:59:30.478ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:38648 collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118
2202025-11-13T07:59:30.478ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 4 n_pruned_tasks = 0
2212025-11-13T07:59:30.479ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
2222025-11-13T07:59:30.479ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
2232025-11-13T07:59:30.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2242025-11-13T07:59:30.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2252025-11-13T07:59:30.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2262025-11-13T07:59:30.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2272025-11-13T07:59:30.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2282025-11-13T07:59:30.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2292025-11-13T07:59:30.510ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
2302025-11-13T07:59:30.532ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
2312025-11-13T07:59:31.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2322025-11-13T07:59:31.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2332025-11-13T07:59:31.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2342025-11-13T07:59:31.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2352025-11-13T07:59:31.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2362025-11-13T07:59:31.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2372025-11-13T07:59:32.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2382025-11-13T07:59:32.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2392025-11-13T07:59:32.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2402025-11-13T07:59:32.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2412025-11-13T07:59:32.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2422025-11-13T07:59:32.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2432025-11-13T07:59:33.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2442025-11-13T07:59:33.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2452025-11-13T07:59:33.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2462025-11-13T07:59:33.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2472025-11-13T07:59:33.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2482025-11-13T07:59:33.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2492025-11-13T07:59:34.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2502025-11-13T07:59:34.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2512025-11-13T07:59:34.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2522025-11-13T07:59:34.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2532025-11-13T07:59:34.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2542025-11-13T07:59:34.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2552025-11-13T07:59:35.220ZDEBGoximeter (results-sink): inserting 477 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
2562025-11-13T07:59:35.220ZDEBGoximeter (results-sink-cluster): inserting 477 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
2572025-11-13T07:59:35.220ZDEBGoximeter (clickhouse-client): unrolling 477 total samples collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
2582025-11-13T07:59:35.222ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
2592025-11-13T07:59:35.260ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:errors_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:35.222289339Z } timeseries_name = switch_port_control_data_link:errors_received
2602025-11-13T07:59:35.260ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
2612025-11-13T07:59:35.262ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:errors_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:35.260283566Z } timeseries_name = switch_port_control_data_link:errors_sent
2622025-11-13T07:59:35.262ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
2632025-11-13T07:59:35.263ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:bytes_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:35.262280465Z } timeseries_name = switch_port_control_data_link:bytes_received
2642025-11-13T07:59:35.264ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
2652025-11-13T07:59:35.265ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:packets_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:35.264064880Z } timeseries_name = switch_port_control_data_link:packets_received
2662025-11-13T07:59:35.265ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
2672025-11-13T07:59:35.267ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:bytes_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:35.265607599Z } timeseries_name = switch_port_control_data_link:bytes_sent
2682025-11-13T07:59:35.267ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
2692025-11-13T07:59:35.268ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:packets_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-11-13T07:59:35.267126950Z } timeseries_name = switch_port_control_data_link:packets_sent
2702025-11-13T07:59:35.285ZDEBGoximeter (clickhouse-client): inserting 6 new timeseries schema collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
2712025-11-13T07:59:35.292ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 24 table_name = fields_i16
2722025-11-13T07:59:35.293ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 2 table_name = fields_ipaddr
2732025-11-13T07:59:35.296ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 173 table_name = fields_string
2742025-11-13T07:59:35.297ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 14 table_name = fields_u16
2752025-11-13T07:59:35.299ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 24 table_name = fields_u32
2762025-11-13T07:59:35.301ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 12 table_name = fields_u8
2772025-11-13T07:59:35.303ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 94 table_name = fields_uuid
2782025-11-13T07:59:35.304ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 427 table_name = measurements_cumulativeu64
2792025-11-13T07:59:35.306ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 50 table_name = measurements_u64
2802025-11-13T07:59:35.320ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
2812025-11-13T07:59:35.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
2822025-11-13T07:59:35.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
2832025-11-13T07:59:35.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2842025-11-13T07:59:35.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2852025-11-13T07:59:35.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2862025-11-13T07:59:35.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2872025-11-13T07:59:35.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2882025-11-13T07:59:35.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2892025-11-13T07:59:35.489ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 5 producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
2902025-11-13T07:59:36.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2912025-11-13T07:59:36.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2922025-11-13T07:59:36.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2932025-11-13T07:59:36.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2942025-11-13T07:59:36.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2952025-11-13T07:59:36.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2962025-11-13T07:59:37.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2972025-11-13T07:59:37.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2982025-11-13T07:59:37.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2992025-11-13T07:59:37.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3002025-11-13T07:59:37.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3012025-11-13T07:59:37.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3022025-11-13T07:59:38.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3032025-11-13T07:59:38.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3042025-11-13T07:59:38.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3052025-11-13T07:59:38.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3062025-11-13T07:59:38.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3072025-11-13T07:59:38.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3082025-11-13T07:59:39.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3092025-11-13T07:59:39.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3102025-11-13T07:59:39.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3112025-11-13T07:59:39.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3122025-11-13T07:59:39.486ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3132025-11-13T07:59:39.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3142025-11-13T07:59:40.220ZDEBGoximeter (results-sink): inserting 232 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
3152025-11-13T07:59:40.220ZDEBGoximeter (results-sink-cluster): inserting 232 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
3162025-11-13T07:59:40.220ZDEBGoximeter (clickhouse-client): unrolling 232 total samples collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
3172025-11-13T07:59:40.231ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 104 table_name = fields_string
3182025-11-13T07:59:40.232ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 26 table_name = fields_u16
3192025-11-13T07:59:40.234ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 87 table_name = fields_uuid
3202025-11-13T07:59:40.236ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 150 table_name = measurements_cumulativeu64
3212025-11-13T07:59:40.238ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 26 table_name = measurements_histogramu64
3222025-11-13T07:59:40.240ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 6 table_name = measurements_i64
3232025-11-13T07:59:40.242ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 50 table_name = measurements_u64
3242025-11-13T07:59:40.321ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
3252025-11-13T07:59:40.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3262025-11-13T07:59:40.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3272025-11-13T07:59:40.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3282025-11-13T07:59:40.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3292025-11-13T07:59:40.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3302025-11-13T07:59:40.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3312025-11-13T07:59:41.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3322025-11-13T07:59:41.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3332025-11-13T07:59:41.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3342025-11-13T07:59:41.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3352025-11-13T07:59:41.486ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3362025-11-13T07:59:41.486ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3372025-11-13T07:59:42.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3382025-11-13T07:59:42.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3392025-11-13T07:59:42.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3402025-11-13T07:59:42.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3412025-11-13T07:59:42.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3422025-11-13T07:59:42.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3432025-11-13T07:59:43.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3442025-11-13T07:59:43.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3452025-11-13T07:59:43.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3462025-11-13T07:59:43.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3472025-11-13T07:59:43.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3482025-11-13T07:59:43.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3492025-11-13T07:59:44.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3502025-11-13T07:59:44.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3512025-11-13T07:59:44.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3522025-11-13T07:59:44.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3532025-11-13T07:59:44.486ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3542025-11-13T07:59:44.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3552025-11-13T07:59:45.219ZDEBGoximeter (results-sink): inserting 200 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
3562025-11-13T07:59:45.219ZDEBGoximeter (results-sink-cluster): inserting 200 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
3572025-11-13T07:59:45.219ZDEBGoximeter (clickhouse-client): unrolling 200 total samples collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
3582025-11-13T07:59:45.228ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 52 table_name = fields_string
3592025-11-13T07:59:45.230ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 56 table_name = fields_uuid
3602025-11-13T07:59:45.232ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 150 table_name = measurements_cumulativeu64
3612025-11-13T07:59:45.234ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 50 table_name = measurements_u64
3622025-11-13T07:59:45.320ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
3632025-11-13T07:59:45.472ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
3642025-11-13T07:59:45.472ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
3652025-11-13T07:59:45.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
3662025-11-13T07:59:45.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
3672025-11-13T07:59:45.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3682025-11-13T07:59:45.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3692025-11-13T07:59:45.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3702025-11-13T07:59:45.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3712025-11-13T07:59:45.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3722025-11-13T07:59:45.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3732025-11-13T07:59:45.489ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 5 producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
3742025-11-13T07:59:45.501ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "a18f1534-894f-4e94-8466-7d858154e710", "content-length": "710", "date": "Thu, 13 Nov 2025 07:59:45 GMT"} })
3752025-11-13T07:59:45.501ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiZTkwZTUxNi04MjRlLTRmZmEtOGIzNC05N2E3OTRmZDcxMTgifX0%3D
3762025-11-13T07:59:45.512ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiZTkwZTUxNi04MjRlLTRmZmEtOGIzNC05N2E3OTRmZDcxMTgifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6688a99-9ab0-43cf-aa88-0c43c9c1f996", "content-length": "29", "date": "Thu, 13 Nov 2025 07:59:45 GMT"} })
3772025-11-13T07:59:45.512ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 4 n_pruned_tasks = 0
3782025-11-13T07:59:46.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3792025-11-13T07:59:46.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3802025-11-13T07:59:46.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3812025-11-13T07:59:46.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3822025-11-13T07:59:46.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3832025-11-13T07:59:46.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3842025-11-13T07:59:47.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3852025-11-13T07:59:47.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3862025-11-13T07:59:47.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3872025-11-13T07:59:47.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3882025-11-13T07:59:47.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3892025-11-13T07:59:47.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3902025-11-13T07:59:48.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3912025-11-13T07:59:48.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3922025-11-13T07:59:48.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3932025-11-13T07:59:48.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3942025-11-13T07:59:48.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3952025-11-13T07:59:48.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3962025-11-13T07:59:49.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3972025-11-13T07:59:49.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3982025-11-13T07:59:49.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3992025-11-13T07:59:49.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4002025-11-13T07:59:49.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4012025-11-13T07:59:49.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4022025-11-13T07:59:50.220ZDEBGoximeter (results-sink): inserting 226 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
4032025-11-13T07:59:50.220ZDEBGoximeter (results-sink-cluster): inserting 226 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
4042025-11-13T07:59:50.220ZDEBGoximeter (clickhouse-client): unrolling 226 total samples collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
4052025-11-13T07:59:50.230ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 104 table_name = fields_string
4062025-11-13T07:59:50.232ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 26 table_name = fields_u16
4072025-11-13T07:59:50.234ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 82 table_name = fields_uuid
4082025-11-13T07:59:50.236ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 150 table_name = measurements_cumulativeu64
4092025-11-13T07:59:50.238ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 26 table_name = measurements_histogramu64
4102025-11-13T07:59:50.240ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 50 table_name = measurements_u64
4112025-11-13T07:59:50.322ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
4122025-11-13T07:59:50.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4132025-11-13T07:59:50.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4142025-11-13T07:59:50.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4152025-11-13T07:59:50.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4162025-11-13T07:59:50.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4172025-11-13T07:59:50.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4182025-11-13T07:59:51.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4192025-11-13T07:59:51.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4202025-11-13T07:59:51.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4212025-11-13T07:59:51.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4222025-11-13T07:59:51.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4232025-11-13T07:59:51.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4242025-11-13T07:59:52.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4252025-11-13T07:59:52.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4262025-11-13T07:59:52.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4272025-11-13T07:59:52.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4282025-11-13T07:59:52.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4292025-11-13T07:59:52.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4302025-11-13T07:59:53.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4312025-11-13T07:59:53.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4322025-11-13T07:59:53.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4332025-11-13T07:59:53.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4342025-11-13T07:59:53.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4352025-11-13T07:59:53.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4362025-11-13T07:59:54.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4372025-11-13T07:59:54.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4382025-11-13T07:59:54.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4392025-11-13T07:59:54.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4402025-11-13T07:59:54.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4412025-11-13T07:59:54.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4422025-11-13T07:59:55.219ZDEBGoximeter (results-sink): inserting 200 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
4432025-11-13T07:59:55.219ZDEBGoximeter (results-sink-cluster): inserting 200 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
4442025-11-13T07:59:55.219ZDEBGoximeter (clickhouse-client): unrolling 200 total samples collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
4452025-11-13T07:59:55.229ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 52 table_name = fields_string
4462025-11-13T07:59:55.231ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 56 table_name = fields_uuid
4472025-11-13T07:59:55.233ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 150 table_name = measurements_cumulativeu64
4482025-11-13T07:59:55.234ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 50 table_name = measurements_u64
4492025-11-13T07:59:55.321ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
4502025-11-13T07:59:55.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
4512025-11-13T07:59:55.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4522025-11-13T07:59:55.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
4532025-11-13T07:59:55.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4542025-11-13T07:59:55.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4552025-11-13T07:59:55.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4562025-11-13T07:59:55.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4572025-11-13T07:59:55.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4582025-11-13T07:59:55.489ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 5 producer_id = 6564a4a0-ff58-4081-9860-74f24e488435 producer_ip = fd00:1122:3344:101::b producer_port = 62328
4592025-11-13T07:59:56.484ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4602025-11-13T07:59:56.484ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4612025-11-13T07:59:56.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4622025-11-13T07:59:56.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4632025-11-13T07:59:56.486ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4642025-11-13T07:59:56.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4652025-11-13T07:59:57.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4662025-11-13T07:59:57.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4672025-11-13T07:59:57.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4682025-11-13T07:59:57.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4692025-11-13T07:59:57.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4702025-11-13T07:59:57.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4712025-11-13T07:59:58.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4722025-11-13T07:59:58.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4732025-11-13T07:59:58.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4742025-11-13T07:59:58.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4752025-11-13T07:59:58.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4762025-11-13T07:59:58.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4772025-11-13T07:59:59.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4782025-11-13T07:59:59.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4792025-11-13T07:59:59.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4802025-11-13T07:59:59.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4812025-11-13T07:59:59.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4822025-11-13T07:59:59.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4832025-11-13T08:00:00.220ZDEBGoximeter (results-sink): inserting 226 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
4842025-11-13T08:00:00.220ZDEBGoximeter (results-sink-cluster): inserting 226 samples into database collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d
4852025-11-13T08:00:00.220ZDEBGoximeter (clickhouse-client): unrolling 226 total samples collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15
4862025-11-13T08:00:00.231ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 104 table_name = fields_string
4872025-11-13T08:00:00.233ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 26 table_name = fields_u16
4882025-11-13T08:00:00.236ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 82 table_name = fields_uuid
4892025-11-13T08:00:00.238ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 150 table_name = measurements_cumulativeu64
4902025-11-13T08:00:00.241ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 26 table_name = measurements_histogramu64
4912025-11-13T08:00:00.242ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d id = d44d737b-0153-48a6-bcaf-41cbb7231e15 n_rows = 50 table_name = measurements_u64
4922025-11-13T08:00:00.321ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
4932025-11-13T08:00:00.473ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
4942025-11-13T08:00:00.473ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
4952025-11-13T08:00:00.477ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "302de5a2-1396-4303-b899-9f6308190f94", "content-length": "859", "date": "Thu, 13 Nov 2025 08:00:00 GMT"} })
4962025-11-13T08:00:00.477ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
4972025-11-13T08:00:00.480ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
4982025-11-13T08:00:00.480ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
4992025-11-13T08:00:00.480ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "87aefe2c-73c4-41d1-af4a-039eaebc939b", "content-length": "29", "date": "Thu, 13 Nov 2025 08:00:00 GMT"} })
5002025-11-13T08:00:00.480ZDEBGoximeter (oximeter-agent): received request to register existing metric producer, updating collection information address = [fd00:1122:3344:101::2]:37826 collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d interval = 10s producer_id = be90e516-824e-4ffa-8b34-97a794fd7118
5012025-11-13T08:00:00.480ZDEBGoximeter (collection-task-handle): notified collection task of new producer endpoint collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
5022025-11-13T08:00:00.480ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:59109 collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = f2c4cd29-0fbf-4648-953f-f8b88934ebc1
5032025-11-13T08:00:00.480ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 5 n_pruned_tasks = 0
5042025-11-13T08:00:00.480ZDEBGoximeter (collection-task): collection task received request to update its producer information address = [fd00:1122:3344:101::2]:37826 collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d interval = 10s producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
5052025-11-13T08:00:00.480ZDEBGoximeter (collection-task): received producer info update with an outstanding collection running, cancelling it and recreating with the new info collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d new_info = ProducerEndpoint { id: be90e516-824e-4ffa-8b34-97a794fd7118, kind: Service, address: [fd00:1122:3344:101::2]:37826, interval: 10s } producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
5062025-11-13T08:00:00.480ZDEBGoximeter (collection-task): collecting from producer address = [fd00:1122:3344:101::2]:37826 collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
5072025-11-13T08:00:00.482ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
5082025-11-13T08:00:00.482ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = f2c4cd29-0fbf-4648-953f-f8b88934ebc1 producer_ip = fd00:1122:3344:101::2 producer_port = 59109
5092025-11-13T08:00:00.482ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = f2c4cd29-0fbf-4648-953f-f8b88934ebc1 producer_ip = fd00:1122:3344:101::2 producer_port = 59109
5102025-11-13T08:00:00.485ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5112025-11-13T08:00:00.485ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5122025-11-13T08:00:00.486ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5132025-11-13T08:00:00.486ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5142025-11-13T08:00:00.487ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 53068167-26e0-432a-b0d6-b9e4fb6d2d2f producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5152025-11-13T08:00:00.488ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8916ef7e-b9e8-4c49-85d7-666251aab81d producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5162025-11-13T08:00:00.514ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = f2c4cd29-0fbf-4648-953f-f8b88934ebc1 producer_ip = fd00:1122:3344:101::2 producer_port = 59109
5172025-11-13T08:00:00.514ZDEBGoximeter (collection-task): collected results from producer address = [fd00:1122:3344:101::2]:37826 collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
5182025-11-13T08:00:00.514ZDEBGoximeter (collection-task): collecting from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648
5192025-11-13T08:00:00.529ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 2 producer_id = f2c4cd29-0fbf-4648-953f-f8b88934ebc1 producer_ip = fd00:1122:3344:101::2 producer_port = 59109
5202025-11-13T08:00:00.535ZDEBGoximeter (collection-task): collected results from producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = be90e516-824e-4ffa-8b34-97a794fd7118 producer_ip = fd00:1122:3344:101::2 producer_port = 38648