Max level shown:
1 [ Dec 15 01:12:56 Disabled. ]
2 [ Dec 15 01:12:56 Rereading configuration. ]
3 [ Dec 15 01:13:14 Rereading configuration. ]
4 [ Dec 15 01:13:14 Enabled. ]
5 [ Dec 15 01:13:14 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/oximeter/bin/oximeter run /var/svc/manifest/site/oximeter/config.toml --address [fd00:1122:3344:101::d]:12223 --id 127efcba-5993-4577-bcb2-f4a98a4520bb &"). ]
6 [ Dec 15 01:13:14 Method "start" exited with status 0. ]
7 note: configured to log to "/dev/stdout"
82023-12-15T01:13:14.890ZDEBGoximeter: registered DTrace probes
92023-12-15T01:13:14.891ZINFOoximeter: starting oximeter server file = oximeter/collector/src/lib.rs:192
102023-12-15T01:13:14.894ZINFOoximeter (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53, [fd00:1122:3344:4::1]:53, [fd00:1122:3344:5::1]:53] file = internal-dns/src/resolver.rs:60
112023-12-15T01:13:14.896ZDEBGoximeter: creating ClickHouse client
122023-12-15T01:13:14.896ZDEBGoximeter (DnsResolver): lookup srv dns_name = _clickhouse._tcp.control-plane.oxide.internal
132023-12-15T01:13:15.089ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb file = oximeter/db/src/client.rs:864 id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
142023-12-15T01:13:15.089ZDEBGoximeter (oximeter-agent): oximeter database does not exist, creating collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb
152023-12-15T01:13:15.096ZINFOoximeter (clickhouse-client): reading db version collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb file = oximeter/db/src/client.rs:802 id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
162023-12-15T01:13:15.097ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb file = oximeter/db/src/client.rs:864 id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
172023-12-15T01:13:15.097ZINFOoximeter (clickhouse-client): read oximeter database version collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb file = oximeter/db/src/client.rs:806 id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e version = 0
182023-12-15T01:13:15.097ZINFOoximeter (clickhouse-client): wiping and re-initializing oximeter schema collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb file = oximeter/db/src/client.rs:811 id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
192023-12-15T01:13:15.097ZDEBGoximeter (clickhouse-client): wiping ClickHouse database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
202023-12-15T01:13:15.098ZDEBGoximeter (clickhouse-client): initializing ClickHouse database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
212023-12-15T01:13:16.478ZINFOoximeter (clickhouse-client): inserting current version collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb file = oximeter/db/src/client.rs:834 id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e version = 4
222023-12-15T01:13:16.487ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /info
232023-12-15T01:13:16.487ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers
242023-12-15T01:13:16.487ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = POST path = /producers
252023-12-15T01:13:16.487ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = DELETE path = /producers/{producer_id}
262023-12-15T01:13:16.487ZINFOoximeter (dropshot): listening file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:195 local_addr = [fd00:1122:3344:101::d]:12223
272023-12-15T01:13:16.487ZDEBGoximeter (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::d]:12223
282023-12-15T01:13:16.520ZDEBGoximeter: contacting nexus
292023-12-15T01:13:16.520ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
302023-12-15T01:13:33.430ZWARNoximeter: failed to contact nexus, will retry in 309.821243ms error = "error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
312023-12-15T01:13:33.740ZDEBGoximeter: contacting nexus
322023-12-15T01:13:33.740ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
332023-12-15T01:13:33.741ZWARNoximeter: failed to contact nexus, will retry in 434.422444ms error = "error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
342023-12-15T01:13:34.177ZDEBGoximeter: contacting nexus
352023-12-15T01:13:34.177ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
362023-12-15T01:13:34.178ZWARNoximeter: failed to contact nexus, will retry in 1.218797942s error = "error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
372023-12-15T01:13:35.398ZDEBGoximeter: contacting nexus
382023-12-15T01:13:35.398ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
392023-12-15T01:13:35.398ZWARNoximeter: failed to contact nexus, will retry in 1.326785139s error = "error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
402023-12-15T01:13:36.727ZDEBGoximeter: contacting nexus
412023-12-15T01:13:36.727ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
422023-12-15T01:13:36.727ZWARNoximeter: failed to contact nexus, will retry in 5.906770369s error = "error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
432023-12-15T01:13:42.636ZDEBGoximeter: contacting nexus
442023-12-15T01:13:42.636ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
452023-12-15T01:13:42.636ZWARNoximeter: failed to contact nexus, will retry in 10.905365639s error = "error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors): error trying to connect: tcp connect error: Connection refused (os error 146)" file = oximeter/collector/src/lib.rs:271
462023-12-15T01:13:53.543ZDEBGoximeter: contacting nexus
472023-12-15T01:13:53.543ZDEBGoximeter (DnsResolver): lookup_ipv6 srv dns_name = _nexus._tcp.control-plane.oxide.internal
482023-12-15T01:13:53.595ZINFOoximeter: oximeter registered with nexus file = oximeter/collector/src/lib.rs:285 id = 127efcba-5993-4577-bcb2-f4a98a4520bb
492023-12-15T01:14:06.101ZINFOoximeter (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::d]:12223 remote_addr = [fd00:1122:3344:101::c]:40984
502023-12-15T01:14:06.108ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::c]:12221 collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
512023-12-15T01:14:06.109ZINFOoximeter (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 7118 local_addr = [fd00:1122:3344:101::d]:12223 method = POST remote_addr = [fd00:1122:3344:101::c]:40984 req_id = 4a45edfc-5fc5-433b-b0b4-43b5de84028a response_code = 204 uri = /producers
522023-12-15T01:14:06.142ZDEBGoximeter (collection-task): starting oximeter collection task collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb interval = 10s producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
532023-12-15T01:14:16.142ZDEBGoximeter (collection-task): collecting from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
542023-12-15T01:14:16.147ZDEBGoximeter (collection-task): collected results from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb n_results = 4 producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
552023-12-15T01:14:16.488ZDEBGoximeter (results-sink): inserting 32 samples into database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb
562023-12-15T01:14:16.488ZDEBGoximeter (clickhouse-client): unrolling 32 total samples collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
572023-12-15T01:14:16.495ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
582023-12-15T01:14:16.511ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e schema = {"timeseries_name":"http_service:request_latency_histogram","fields.name":["id","method","name","route","status_code"],"fields.type":["Uuid","String","String","String","I64"],"fields.source":["Target","Metric","Target","Metric","Metric"],"datum_type":"HistogramF64","created":"2023-12-15 01:14:16.495552350"} timeseries_name = http_service:request_latency_histogram
592023-12-15T01:14:16.516ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
602023-12-15T01:14:16.531ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e schema = {"timeseries_name":"collection_target:cpus_provisioned","fields.name":["id"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"I64","created":"2023-12-15 01:14:16.516339983"} timeseries_name = collection_target:cpus_provisioned
612023-12-15T01:14:16.531ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
622023-12-15T01:14:16.535ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e schema = {"timeseries_name":"collection_target:ram_provisioned","fields.name":["id"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"I64","created":"2023-12-15 01:14:16.531545682"} timeseries_name = collection_target:ram_provisioned
632023-12-15T01:14:16.535ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
642023-12-15T01:14:16.537ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e schema = {"timeseries_name":"collection_target:virtual_disk_space_provisioned","fields.name":["id"],"fields.type":["Uuid"],"fields.source":["Target"],"datum_type":"I64","created":"2023-12-15 01:14:16.535558743"} timeseries_name = collection_target:virtual_disk_space_provisioned
652023-12-15T01:14:16.537ZDEBGoximeter (clickhouse-client): inserting 4 new timeseries schema collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
662023-12-15T01:14:16.551ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 20 table_name = oximeter.fields_i64
672023-12-15T01:14:16.552ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 60 table_name = oximeter.fields_string
682023-12-15T01:14:16.557ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 27 table_name = oximeter.fields_uuid
692023-12-15T01:14:16.561ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 20 table_name = oximeter.measurements_histogramf64
702023-12-15T01:14:16.563ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 12 table_name = oximeter.measurements_i64
712023-12-15T01:14:26.142ZDEBGoximeter (collection-task): collecting from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
722023-12-15T01:14:26.144ZDEBGoximeter (collection-task): collected results from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb n_results = 4 producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
732023-12-15T01:14:26.488ZDEBGoximeter (results-sink): inserting 22 samples into database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb
742023-12-15T01:14:26.488ZDEBGoximeter (clickhouse-client): unrolling 22 total samples collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
752023-12-15T01:14:26.491ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.fields_i64
762023-12-15T01:14:26.492ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 66 table_name = oximeter.fields_string
772023-12-15T01:14:26.494ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.fields_uuid
782023-12-15T01:14:26.496ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.measurements_histogramf64
792023-12-15T01:14:27.957ZINFOoximeter (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::d]:12223 remote_addr = [fd00:1122:3344:101::b]:60784
802023-12-15T01:14:27.958ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:12345 collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
812023-12-15T01:14:27.958ZINFOoximeter (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 657 local_addr = [fd00:1122:3344:101::d]:12223 method = POST remote_addr = [fd00:1122:3344:101::b]:60784 req_id = 47c65b3e-66db-41df-a6e7-95ce4702b451 response_code = 204 uri = /producers
822023-12-15T01:14:27.991ZDEBGoximeter (collection-task): starting oximeter collection task collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb interval = 30s producer_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
832023-12-15T01:14:36.142ZDEBGoximeter (collection-task): collecting from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
842023-12-15T01:14:36.144ZDEBGoximeter (collection-task): collected results from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb n_results = 4 producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
852023-12-15T01:14:36.488ZDEBGoximeter (results-sink): inserting 22 samples into database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb
862023-12-15T01:14:36.488ZDEBGoximeter (clickhouse-client): unrolling 22 total samples collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
872023-12-15T01:14:36.491ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.fields_i64
882023-12-15T01:14:36.493ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 66 table_name = oximeter.fields_string
892023-12-15T01:14:36.494ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.fields_uuid
902023-12-15T01:14:36.496ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.measurements_histogramf64
912023-12-15T01:14:46.141ZDEBGoximeter (collection-task): collecting from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
922023-12-15T01:14:46.143ZDEBGoximeter (collection-task): collected results from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb n_results = 4 producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
932023-12-15T01:14:46.487ZDEBGoximeter (results-sink): inserting 22 samples into database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb
942023-12-15T01:14:46.487ZDEBGoximeter (clickhouse-client): unrolling 22 total samples collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
952023-12-15T01:14:46.491ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.fields_i64
962023-12-15T01:14:46.492ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 66 table_name = oximeter.fields_string
972023-12-15T01:14:46.493ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.fields_uuid
982023-12-15T01:14:46.495ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.measurements_histogramf64
992023-12-15T01:14:50.026ZINFOoximeter (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::d]:12223 remote_addr = [fd00:1122:3344:101::b]:35958
1002023-12-15T01:14:50.027ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::b]:12221 collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 7198c8dc-786b-4865-a05e-06e4223227c4
1012023-12-15T01:14:50.027ZINFOoximeter (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 113 local_addr = [fd00:1122:3344:101::d]:12223 method = POST remote_addr = [fd00:1122:3344:101::b]:35958 req_id = be28623c-5d5a-409b-b579-5e70efd069bf response_code = 204 uri = /producers
1022023-12-15T01:14:50.059ZDEBGoximeter (collection-task): starting oximeter collection task collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb interval = 10s producer_id = 7198c8dc-786b-4865-a05e-06e4223227c4
1032023-12-15T01:14:50.104ZINFOoximeter (dropshot): accepted connection file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:769 local_addr = [fd00:1122:3344:101::d]:12223 remote_addr = [fd00:1122:3344:101::a]:59504
1042023-12-15T01:14:50.104ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::a]:12221 collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 72d8f37c-a72e-4800-9c47-39f6b4e0d24b
1052023-12-15T01:14:50.104ZINFOoximeter (dropshot): request completed file = /home/build/.cargo/git/checkouts/dropshot-a4a923d29dccc492/ff87a01/dropshot/src/server.rs:853 latency_us = 91 local_addr = [fd00:1122:3344:101::d]:12223 method = POST remote_addr = [fd00:1122:3344:101::a]:59504 req_id = 120aeff1-7e57-46e5-8619-d961aac0445a response_code = 204 uri = /producers
1062023-12-15T01:14:50.137ZDEBGoximeter (collection-task): starting oximeter collection task collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb interval = 10s producer_id = 72d8f37c-a72e-4800-9c47-39f6b4e0d24b
1072023-12-15T01:14:56.142ZDEBGoximeter (collection-task): collecting from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
1082023-12-15T01:14:56.144ZDEBGoximeter (collection-task): collected results from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb n_results = 4 producer_id = 700e7b49-f840-40d5-92e8-83ab758ea8a7
1092023-12-15T01:14:56.487ZDEBGoximeter (results-sink): inserting 22 samples into database collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb
1102023-12-15T01:14:56.487ZDEBGoximeter (clickhouse-client): unrolling 22 total samples collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e
1112023-12-15T01:14:56.490ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.fields_i64
1122023-12-15T01:14:56.492ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 66 table_name = oximeter.fields_string
1132023-12-15T01:14:56.493ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.fields_uuid
1142023-12-15T01:14:56.495ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb id = 299520aa-84a8-4d4e-9d5e-c9a30abd150e n_rows = 22 table_name = oximeter.measurements_histogramf64
1152023-12-15T01:14:57.991ZDEBGoximeter (collection-task): collecting from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
1162023-12-15T01:14:57.997ZDEBGoximeter (collection-task): collected results from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb n_results = 1 producer_id = 78829061-d92a-47ac-bcd6-186a0b9f0582
1172023-12-15T01:15:00.059ZDEBGoximeter (collection-task): collecting from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 7198c8dc-786b-4865-a05e-06e4223227c4
1182023-12-15T01:15:00.061ZDEBGoximeter (collection-task): collected results from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb n_results = 4 producer_id = 7198c8dc-786b-4865-a05e-06e4223227c4
1192023-12-15T01:15:00.138ZDEBGoximeter (collection-task): collecting from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb producer_id = 72d8f37c-a72e-4800-9c47-39f6b4e0d24b
1202023-12-15T01:15:00.141ZDEBGoximeter (collection-task): collected results from producer collector_id = 127efcba-5993-4577-bcb2-f4a98a4520bb n_results = 4 producer_id = 72d8f37c-a72e-4800-9c47-39f6b4e0d24b