Max level shown:
1 [ Nov 18 01:27:47 Disabled. ]
2 [ Nov 18 01:27:47 Rereading configuration. ]
3 [ Nov 18 01:28:15 Rereading configuration. ]
4 [ Nov 18 01:28:15 Rereading configuration. ]
5 [ Nov 18 01:28:15 Enabled. ]
6 [ Nov 18 01:28:15 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/oximeter/bin/oximeter run /var/svc/manifest/site/oximeter/config.toml --address [fd00:1122:3344:101::d]:12223 --id 89b351b3-e9ba-45cd-a0d2-454fcafa702f &"). ]
7 [ Nov 18 01:28:15 Method "start" exited with status 0. ]
8 note: configured to log to "/dev/stdout"
92023-11-18T01:28:15.555ZDEBGoximeter: registered DTrace probes
102023-11-18T01:28:15.561ZINFOoximeter: starting oximeter server file = oximeter/collector/src/lib.rs:192
112023-11-18T01:28:15.567ZINFOoximeter (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
122023-11-18T01:28:15.578ZDEBGoximeter: creating ClickHouse client
132023-11-18T01:28:15.580ZDEBGoximeter (DnsResolver): lookup srv dns_name = _clickhouse._tcp.control-plane.oxide.internal
142023-11-18T01:28:15.720ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f file = oximeter/db/src/client.rs:658 id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
152023-11-18T01:28:15.720ZDEBGoximeter (oximeter-agent): oximeter database does not exist, creating collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
162023-11-18T01:28:15.749ZINFOoximeter (clickhouse-client): reading db version collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f file = oximeter/db/src/client.rs:596 id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
172023-11-18T01:28:15.749ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f file = oximeter/db/src/client.rs:658 id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
182023-11-18T01:28:15.749ZINFOoximeter (clickhouse-client): read oximeter database version collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f file = oximeter/db/src/client.rs:600 id = a24361d0-3c5c-4d2f-856c-07c9fb04564a version = 0
192023-11-18T01:28:15.750ZINFOoximeter (clickhouse-client): wiping and re-initializing oximeter schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f file = oximeter/db/src/client.rs:605 id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
202023-11-18T01:28:15.750ZDEBGoximeter (clickhouse-client): wiping ClickHouse database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
212023-11-18T01:28:15.750ZDEBGoximeter (clickhouse-client): initializing ClickHouse database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
222023-11-18T01:28:15.937ZINFOoximeter (clickhouse-client): inserting current version collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f file = oximeter/db/src/client.rs:628 id = a24361d0-3c5c-4d2f-856c-07c9fb04564a version = 3
232023-11-18T01:28:15.968ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /info
242023-11-18T01:28:15.968ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers
252023-11-18T01:28:15.968ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = POST path = /producers
262023-11-18T01:28:15.968ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = DELETE path = /producers/{producer_id}
272023-11-18T01:28:15.968ZINFOoximeter (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:195 local_addr = [fd00:1122:3344:101::d]:12223
282023-11-18T01:28:15.968ZDEBGoximeter (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::d]:12223
292023-11-18T01:28:16.002ZDEBGoximeter: contacting nexus
302023-11-18T01:28:16.002ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
312023-11-18T01:28:16.003ZWARNoximeter: failed to contact nexus, will retry in 162.625782ms error = "error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
322023-11-18T01:28:16.167ZDEBGoximeter: contacting nexus
332023-11-18T01:28:16.167ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
342023-11-18T01:28:16.168ZWARNoximeter: failed to contact nexus, will retry in 518.357376ms error = "error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
352023-11-18T01:28:16.687ZDEBGoximeter: contacting nexus
362023-11-18T01:28:16.687ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
372023-11-18T01:28:16.688ZWARNoximeter: failed to contact nexus, will retry in 1.350489026s error = "error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
382023-11-18T01:28:18.040ZDEBGoximeter: contacting nexus
392023-11-18T01:28:18.040ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
402023-11-18T01:28:18.041ZWARNoximeter: failed to contact nexus, will retry in 2.23032386s error = "error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
412023-11-18T01:28:20.272ZDEBGoximeter: contacting nexus
422023-11-18T01:28:20.272ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
432023-11-18T01:28:20.273ZWARNoximeter: failed to contact nexus, will retry in 4.306500882s error = "error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
442023-11-18T01:28:24.581ZDEBGoximeter: contacting nexus
452023-11-18T01:28:24.581ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
462023-11-18T01:28:24.788ZINFOoximeter: oximeter registered with nexus file = oximeter/collector/src/lib.rs:285 id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
472023-11-18T01:28:37.826ZINFOoximeter (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::d]:12223 remote_addr = [fd00:1122:3344:101::b]:57619
482023-11-18T01:28:37.826ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:12345 collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b4894a25-ec6c-4b23-bdd3-8424332381f9
492023-11-18T01:28:37.826ZINFOoximeter (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 410 local_addr = [fd00:1122:3344:101::d]:12223 method = POST remote_addr = [fd00:1122:3344:101::b]:57619 req_id = 5d53f3f9-241b-4284-bd6e-972ffcea523b response_code = 204 uri = /producers
502023-11-18T01:28:37.860ZDEBGoximeter (collection-task): starting oximeter collection task collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f interval = 30s producer_id = b4894a25-ec6c-4b23-bdd3-8424332381f9
512023-11-18T01:28:46.969ZINFOoximeter (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::d]:12223 remote_addr = [fd00:1122:3344:101::a]:38018
522023-11-18T01:28:46.970ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::a]:12221 collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
532023-11-18T01:28:46.970ZINFOoximeter (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 716 local_addr = [fd00:1122:3344:101::d]:12223 method = POST remote_addr = [fd00:1122:3344:101::a]:38018 req_id = 7944a1ca-c3b5-4a9a-8550-7786d72e0aba response_code = 204 uri = /producers
542023-11-18T01:28:47.004ZDEBGoximeter (collection-task): starting oximeter collection task collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f interval = 10s producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
552023-11-18T01:28:57.004ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
562023-11-18T01:28:57.006ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
572023-11-18T01:29:00.967ZDEBGoximeter (results-sink): inserting 38 samples into database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
582023-11-18T01:29:00.967ZDEBGoximeter (clickhouse-client): unrolling 38 total samples collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
592023-11-18T01:29:00.970ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
602023-11-18T01:29:00.977ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"http_service:request_latency_histogram","fields.name":["id","method","name","route","status_code"],"fields.type":["Uuid","String","String","String","I64"],"fields.source":["Target","Metric","Target","Metric","Metric"],"datum_type":"HistogramF64","created":"2023-11-18 01:29:00.967935067"} timeseries_name = http_service:request_latency_histogram
612023-11-18T01:29:00.979ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
622023-11-18T01:29:00.995ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"collection_target:cpus_provisioned","fields.name":["id"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"I64","created":"2023-11-18 01:29:00.979065605"} timeseries_name = collection_target:cpus_provisioned
632023-11-18T01:29:00.995ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
642023-11-18T01:29:00.999ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"collection_target:ram_provisioned","fields.name":["id"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"I64","created":"2023-11-18 01:29:00.995515361"} timeseries_name = collection_target:ram_provisioned
652023-11-18T01:29:01.000ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
662023-11-18T01:29:01.002ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"collection_target:virtual_disk_space_provisioned","fields.name":["id"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"I64","created":"2023-11-18 01:29:00.999784228"} timeseries_name = collection_target:virtual_disk_space_provisioned
672023-11-18T01:29:01.003ZDEBGoximeter (clickhouse-client): inserting 4 new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
682023-11-18T01:29:01.020ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 14 table_name = oximeter.fields_i64
692023-11-18T01:29:01.021ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 42 table_name = oximeter.fields_string
702023-11-18T01:29:01.027ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 26 table_name = oximeter.fields_uuid
712023-11-18T01:29:01.033ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 14 table_name = oximeter.measurements_histogramf64
722023-11-18T01:29:01.035ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 24 table_name = oximeter.measurements_i64
732023-11-18T01:29:03.963ZINFOoximeter (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::d]:12223 remote_addr = [fd00:1122:3344:101::c]:44590
742023-11-18T01:29:03.963ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::22]:45159 collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b98269e4-f94c-4e46-99be-f9a2d87a3763
752023-11-18T01:29:03.963ZINFOoximeter (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 531 local_addr = [fd00:1122:3344:101::d]:12223 method = POST remote_addr = [fd00:1122:3344:101::c]:44590 req_id = d07cf177-1502-499b-9789-2f98a7c5f585 response_code = 204 uri = /producers
762023-11-18T01:29:03.997ZDEBGoximeter (collection-task): starting oximeter collection task collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f interval = 30s producer_id = b98269e4-f94c-4e46-99be-f9a2d87a3763
772023-11-18T01:29:07.004ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
782023-11-18T01:29:07.006ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
792023-11-18T01:29:07.860ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b4894a25-ec6c-4b23-bdd3-8424332381f9
802023-11-18T01:29:07.867ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 1 producer_id = b4894a25-ec6c-4b23-bdd3-8424332381f9
812023-11-18T01:29:10.967ZDEBGoximeter (results-sink): inserting 414 samples into database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
822023-11-18T01:29:10.967ZDEBGoximeter (clickhouse-client): unrolling 414 total samples collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
832023-11-18T01:29:10.969ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
842023-11-18T01:29:10.993ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"physical_data_link:errors_received","fields.name":["hostname","link_name","rack_id","serial","sled_id"],"fields.type":["String","String","Uuid","String","Uuid"],"fields.source":["Target","Target","Target","Target","Target"],"datum_type":"CumulativeU64","created":"2023-11-18 01:29:10.969371338"} timeseries_name = physical_data_link:errors_received
852023-11-18T01:29:10.993ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
862023-11-18T01:29:10.995ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"physical_data_link:errors_sent","fields.name":["hostname","link_name","rack_id","serial","sled_id"],"fields.type":["String","String","Uuid","String","Uuid"],"fields.source":["Target","Target","Target","Target","Target"],"datum_type":"CumulativeU64","created":"2023-11-18 01:29:10.993148904"} timeseries_name = physical_data_link:errors_sent
872023-11-18T01:29:10.995ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
882023-11-18T01:29:10.998ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"physical_data_link:bytes_received","fields.name":["hostname","link_name","rack_id","serial","sled_id"],"fields.type":["String","String","Uuid","String","Uuid"],"fields.source":["Target","Target","Target","Target","Target"],"datum_type":"CumulativeU64","created":"2023-11-18 01:29:10.995622065"} timeseries_name = physical_data_link:bytes_received
892023-11-18T01:29:10.998ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
902023-11-18T01:29:11.001ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"physical_data_link:packets_received","fields.name":["hostname","link_name","rack_id","serial","sled_id"],"fields.type":["String","String","Uuid","String","Uuid"],"fields.source":["Target","Target","Target","Target","Target"],"datum_type":"CumulativeU64","created":"2023-11-18 01:29:10.998293549"} timeseries_name = physical_data_link:packets_received
912023-11-18T01:29:11.001ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
922023-11-18T01:29:11.003ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"physical_data_link:bytes_sent","fields.name":["hostname","link_name","rack_id","serial","sled_id"],"fields.type":["String","String","Uuid","String","Uuid"],"fields.source":["Target","Target","Target","Target","Target"],"datum_type":"CumulativeU64","created":"2023-11-18 01:29:11.001246540"} timeseries_name = physical_data_link:bytes_sent
932023-11-18T01:29:11.003ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
942023-11-18T01:29:11.006ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"physical_data_link:packets_sent","fields.name":["hostname","link_name","rack_id","serial","sled_id"],"fields.type":["String","String","Uuid","String","Uuid"],"fields.source":["Target","Target","Target","Target","Target"],"datum_type":"CumulativeU64","created":"2023-11-18 01:29:11.003768541"} timeseries_name = physical_data_link:packets_sent
952023-11-18T01:29:11.025ZDEBGoximeter (clickhouse-client): inserting 6 new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
962023-11-18T01:29:11.028ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.fields_i64
972023-11-18T01:29:11.030ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 90 table_name = oximeter.fields_string
982023-11-18T01:29:11.031ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 42 table_name = oximeter.fields_uuid
992023-11-18T01:29:11.033ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 396 table_name = oximeter.measurements_cumulativeu64
1002023-11-18T01:29:11.035ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.measurements_histogramf64
1012023-11-18T01:29:17.004ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
1022023-11-18T01:29:17.006ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
1032023-11-18T01:29:20.967ZDEBGoximeter (results-sink): inserting 18 samples into database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
1042023-11-18T01:29:20.967ZDEBGoximeter (clickhouse-client): unrolling 18 total samples collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1052023-11-18T01:29:20.971ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.fields_i64
1062023-11-18T01:29:20.973ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 54 table_name = oximeter.fields_string
1072023-11-18T01:29:20.974ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.fields_uuid
1082023-11-18T01:29:20.976ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.measurements_histogramf64
1092023-11-18T01:29:22.913ZINFOoximeter (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::d]:12223 remote_addr = [fd00:1122:3344:101::c]:39785
1102023-11-18T01:29:22.913ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::c]:12221 collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = e6b79618-15a5-443b-836a-645697a3e8db
1112023-11-18T01:29:22.913ZINFOoximeter (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 110 local_addr = [fd00:1122:3344:101::d]:12223 method = POST remote_addr = [fd00:1122:3344:101::c]:39785 req_id = 1bbf6309-c2f4-4a63-8dc3-2c757345aff4 response_code = 204 uri = /producers
1122023-11-18T01:29:22.947ZDEBGoximeter (collection-task): starting oximeter collection task collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f interval = 10s producer_id = e6b79618-15a5-443b-836a-645697a3e8db
1132023-11-18T01:29:23.080ZINFOoximeter (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::d]:12223 remote_addr = [fd00:1122:3344:101::b]:37397
1142023-11-18T01:29:23.080ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::b]:12221 collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = bef2b8ae-5644-4674-b728-2a4b9cd96f85
1152023-11-18T01:29:23.080ZINFOoximeter (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 93 local_addr = [fd00:1122:3344:101::d]:12223 method = POST remote_addr = [fd00:1122:3344:101::b]:37397 req_id = 6f62f4d7-fe4b-44c0-983e-94f4810f089f response_code = 204 uri = /producers
1162023-11-18T01:29:23.114ZDEBGoximeter (collection-task): starting oximeter collection task collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f interval = 10s producer_id = bef2b8ae-5644-4674-b728-2a4b9cd96f85
1172023-11-18T01:29:27.004ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
1182023-11-18T01:29:27.006ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
1192023-11-18T01:29:30.967ZDEBGoximeter (results-sink): inserting 18 samples into database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
1202023-11-18T01:29:30.967ZDEBGoximeter (clickhouse-client): unrolling 18 total samples collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1212023-11-18T01:29:30.970ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.fields_i64
1222023-11-18T01:29:30.972ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 54 table_name = oximeter.fields_string
1232023-11-18T01:29:30.973ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.fields_uuid
1242023-11-18T01:29:30.975ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.measurements_histogramf64
1252023-11-18T01:29:32.947ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = e6b79618-15a5-443b-836a-645697a3e8db
1262023-11-18T01:29:32.949ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = e6b79618-15a5-443b-836a-645697a3e8db
1272023-11-18T01:29:33.115ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = bef2b8ae-5644-4674-b728-2a4b9cd96f85
1282023-11-18T01:29:33.118ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = bef2b8ae-5644-4674-b728-2a4b9cd96f85
1292023-11-18T01:29:33.998ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b98269e4-f94c-4e46-99be-f9a2d87a3763
1302023-11-18T01:29:33.999ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 2 producer_id = b98269e4-f94c-4e46-99be-f9a2d87a3763
1312023-11-18T01:29:35.967ZDEBGoximeter (results-sink): inserting 46 samples into database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
1322023-11-18T01:29:35.967ZDEBGoximeter (clickhouse-client): unrolling 46 total samples collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1332023-11-18T01:29:35.970ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1342023-11-18T01:29:35.974ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"instance_uuid:reset","fields.name":["uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:35.970365556"} timeseries_name = instance_uuid:reset
1352023-11-18T01:29:35.974ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1362023-11-18T01:29:35.978ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:activated","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:35.974598953"} timeseries_name = crucible_upstairs:activated
1372023-11-18T01:29:35.978ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1382023-11-18T01:29:35.982ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:flush","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:35.978427861"} timeseries_name = crucible_upstairs:flush
1392023-11-18T01:29:35.982ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1402023-11-18T01:29:35.985ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:write","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:35.982209218"} timeseries_name = crucible_upstairs:write
1412023-11-18T01:29:35.985ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1422023-11-18T01:29:35.989ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:write_bytes","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:35.985655659"} timeseries_name = crucible_upstairs:write_bytes
1432023-11-18T01:29:35.989ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1442023-11-18T01:29:35.992ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:read","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:35.989079429"} timeseries_name = crucible_upstairs:read
1452023-11-18T01:29:35.992ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1462023-11-18T01:29:35.995ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:read_bytes","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:35.992438337"} timeseries_name = crucible_upstairs:read_bytes
1472023-11-18T01:29:35.995ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1482023-11-18T01:29:35.999ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:flush_close","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:35.995811436"} timeseries_name = crucible_upstairs:flush_close
1492023-11-18T01:29:35.999ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1502023-11-18T01:29:36.003ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:extent_repair","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:35.999235136"} timeseries_name = crucible_upstairs:extent_repair
1512023-11-18T01:29:36.003ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1522023-11-18T01:29:36.006ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:extent_no_op","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:36.003106925"} timeseries_name = crucible_upstairs:extent_no_op
1532023-11-18T01:29:36.006ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1542023-11-18T01:29:36.009ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"crucible_upstairs:extent_reopen","fields.name":["upstairs_uuid"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"CumulativeI64","created":"2023-11-18 01:29:36.006491205"} timeseries_name = crucible_upstairs:extent_reopen
1552023-11-18T01:29:36.009ZDEBGoximeter (clickhouse-client): inserting 11 new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1562023-11-18T01:29:36.012ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 26 table_name = oximeter.fields_i64
1572023-11-18T01:29:36.014ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 78 table_name = oximeter.fields_string
1582023-11-18T01:29:36.015ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 46 table_name = oximeter.fields_uuid
1592023-11-18T01:29:36.017ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 11 table_name = oximeter.measurements_cumulativei64
1602023-11-18T01:29:36.018ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 26 table_name = oximeter.measurements_histogramf64
1612023-11-18T01:29:36.020ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 9 table_name = oximeter.measurements_i64
1622023-11-18T01:29:37.004ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
1632023-11-18T01:29:37.006ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
1642023-11-18T01:29:37.860ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b4894a25-ec6c-4b23-bdd3-8424332381f9
1652023-11-18T01:29:37.861ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 1 producer_id = b4894a25-ec6c-4b23-bdd3-8424332381f9
1662023-11-18T01:29:37.861ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b4894a25-ec6c-4b23-bdd3-8424332381f9
1672023-11-18T01:29:40.967ZDEBGoximeter (results-sink): inserting 55 samples into database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
1682023-11-18T01:29:40.967ZDEBGoximeter (clickhouse-client): unrolling 55 total samples collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1692023-11-18T01:29:40.971ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1702023-11-18T01:29:40.976ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a schema = {"timeseries_name":"oximeter_collector:collections","fields.name":["base_route","collector_id","collector_ip","collector_port","producer_id","producer_ip","producer_port"],"fields.type":["String","Uuid","IpAddr","U16","Uuid","IpAddr","U16"],"fields.source":["Metric","Target","Target","Target","Metric","Metric","Metric"],"datum_type":"CumulativeU64","created":"2023-11-18 01:29:40.971172039"} timeseries_name = oximeter_collector:collections
1712023-11-18T01:29:40.976ZDEBGoximeter (clickhouse-client): inserting 1 new timeseries schema collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1722023-11-18T01:29:40.979ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.fields_i64
1732023-11-18T01:29:40.980ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 2 table_name = oximeter.fields_ipaddr
1742023-11-18T01:29:40.982ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 91 table_name = oximeter.fields_string
1752023-11-18T01:29:40.989ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 2 table_name = oximeter.fields_u16
1762023-11-18T01:29:40.990ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 44 table_name = oximeter.fields_uuid
1772023-11-18T01:29:40.994ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 37 table_name = oximeter.measurements_cumulativeu64
1782023-11-18T01:29:40.996ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.measurements_histogramf64
1792023-11-18T01:29:42.948ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = e6b79618-15a5-443b-836a-645697a3e8db
1802023-11-18T01:29:42.949ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = e6b79618-15a5-443b-836a-645697a3e8db
1812023-11-18T01:29:43.114ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = bef2b8ae-5644-4674-b728-2a4b9cd96f85
1822023-11-18T01:29:43.117ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = bef2b8ae-5644-4674-b728-2a4b9cd96f85
1832023-11-18T01:29:45.967ZDEBGoximeter (results-sink): inserting 28 samples into database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
1842023-11-18T01:29:45.967ZDEBGoximeter (clickhouse-client): unrolling 28 total samples collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1852023-11-18T01:29:45.971ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 28 table_name = oximeter.fields_i64
1862023-11-18T01:29:45.973ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 84 table_name = oximeter.fields_string
1872023-11-18T01:29:45.974ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 28 table_name = oximeter.fields_uuid
1882023-11-18T01:29:45.976ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 28 table_name = oximeter.measurements_histogramf64
1892023-11-18T01:29:47.004ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
1902023-11-18T01:29:47.006ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
1912023-11-18T01:29:47.007ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
1922023-11-18T01:29:50.967ZDEBGoximeter (results-sink): inserting 19 samples into database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
1932023-11-18T01:29:50.967ZDEBGoximeter (clickhouse-client): unrolling 19 total samples collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
1942023-11-18T01:29:50.970ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.fields_i64
1952023-11-18T01:29:50.972ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 2 table_name = oximeter.fields_ipaddr
1962023-11-18T01:29:50.973ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 55 table_name = oximeter.fields_string
1972023-11-18T01:29:50.975ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 2 table_name = oximeter.fields_u16
1982023-11-18T01:29:50.976ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 20 table_name = oximeter.fields_uuid
1992023-11-18T01:29:50.977ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 1 table_name = oximeter.measurements_cumulativeu64
2002023-11-18T01:29:50.979ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 18 table_name = oximeter.measurements_histogramf64
2012023-11-18T01:29:52.948ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = e6b79618-15a5-443b-836a-645697a3e8db
2022023-11-18T01:29:52.948ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = e6b79618-15a5-443b-836a-645697a3e8db
2032023-11-18T01:29:53.115ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = bef2b8ae-5644-4674-b728-2a4b9cd96f85
2042023-11-18T01:29:53.174ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = bef2b8ae-5644-4674-b728-2a4b9cd96f85
2052023-11-18T01:29:55.968ZDEBGoximeter (results-sink): inserting 28 samples into database collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f
2062023-11-18T01:29:55.968ZDEBGoximeter (clickhouse-client): unrolling 28 total samples collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a
2072023-11-18T01:29:55.972ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 28 table_name = oximeter.fields_i64
2082023-11-18T01:29:55.973ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 84 table_name = oximeter.fields_string
2092023-11-18T01:29:55.975ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 28 table_name = oximeter.fields_uuid
2102023-11-18T01:29:55.976ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f id = a24361d0-3c5c-4d2f-856c-07c9fb04564a n_rows = 28 table_name = oximeter.measurements_histogramf64
2112023-11-18T01:29:57.005ZDEBGoximeter (collection-task): collecting from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f
2122023-11-18T01:29:57.006ZDEBGoximeter (collection-task): collected results from producer collector_id = 89b351b3-e9ba-45cd-a0d2-454fcafa702f n_results = 3 producer_id = b2d1fed7-2ddf-4e87-909c-32c8db178f7f