| 1 | 2026.08.17 23:52:05.849200 [ 423881 ] {} <Information> Application: Will watch for the process with pid 423933 | ||
| 2 | 2026.08.17 23:52:05.849261 [ 423933 ] {} <Information> Application: Forked a child process to watch | ||
| 3 | 2026.08.17 23:52:05.849400 [ 423933 ] {} <Information> SentryWriter: Sending crash reports is disabled | ||
| 4 | 2026.08.17 23:52:05.849483 [ 423933 ] {} <Trace> Pipe: Pipe capacity is 1.00 MiB | ||
| 5 | 2026.08.17 23:52:05.885143 [ 423933 ] {} <Information> Application: Starting ClickHouse 23.8.7.24 (revision: 54477, git hash: 812b95e14ba8cf744bf1d70e6de607cf130a79fa, build id: 6484B75ECB16F4FF87A88E10249A45E1BDBDE2E5), PID 423933 | ||
| 6 | 2026.08.17 23:52:05.885918 [ 423933 ] {} <Information> Application: starting up | ||
| 7 | 2026.08.17 23:52:05.885931 [ 423933 ] {} <Information> Application: OS name: Linux, version: 5.15.0-1015-aws, architecture: x86_64 | ||
| 8 | 2026.08.17 23:52:05.887965 [ 423933 ] {} <Information> Application: Available RAM: 30.74 GiB; physical cores: 16; logical cores: 16. | ||
| 9 | 2026.08.17 23:52:05.888344 [ 423933 ] {} <Trace> AsynchronousMetrics: Scanning /sys/class/thermal | ||
| 10 | 2026.08.17 23:52:05.888351 [ 423933 ] {} <Trace> AsynchronousMetrics: Scanning /sys/block | ||
| 11 | 2026.08.17 23:52:05.888366 [ 423933 ] {} <Trace> AsynchronousMetrics: Scanning /sys/devices/system/edac | ||
| 12 | 2026.08.17 23:52:05.888372 [ 423933 ] {} <Trace> AsynchronousMetrics: Scanning /sys/class/hwmon | ||
| 13 | 2026.08.17 23:52:05.993737 [ 423933 ] {} <Information> Application: Integrity check of the executable successfully passed (checksum: 63E397DB90569A93F13765E06F4A1FBF) | ||
| 14 | 2026.08.17 23:52:06.000107 [ 423933 ] {} <Debug> Application: Set max number of file descriptors to 524288 (was 1024). | ||
| 15 | 2026.08.17 23:52:06.000120 [ 423933 ] {} <Debug> Application: rlimit on number of threads is 125877 | ||
| 16 | 2026.08.17 23:52:06.000125 [ 423933 ] {} <Debug> Application: Initializing DateLUT. | ||
| 17 | 2026.08.17 23:52:06.000127 [ 423933 ] {} <Trace> Application: Initialized DateLUT with time zone 'UTC'. | ||
| 18 | 2026.08.17 23:52:06.000140 [ 423933 ] {} <Debug> Context: Setting up /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/data/tmp/ to store temporary data in it | ||
| 19 | 2026.08.17 23:52:06.001385 [ 423933 ] {} <Debug> Application: Initializing interserver credentials. | ||
| 20 | 2026.08.17 23:52:06.001664 [ 423933 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' | ||
| 21 | 2026.08.17 23:52:06.002009 [ 423933 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 22 | 2026.08.17 23:52:06.002297 [ 423933 ] {} <Information> Application: Setting max_server_memory_usage was set to 27.67 GiB (30.74 GiB available * 0.90 max_server_memory_usage_to_ram_ratio) | ||
| 23 | 2026.08.17 23:52:06.002300 [ 423933 ] {} <Information> Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.37 GiB (30.74 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) | ||
| 24 | 2026.08.17 23:52:06.002302 [ 423933 ] {} <Information> Application: Merges and mutations memory limit is set to 15.37 GiB | ||
| 25 | 2026.08.17 23:52:06.002452 [ 423933 ] {} <Information> BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 26 | 2026.08.17 23:52:06.002813 [ 423933 ] {} <Information> BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 128 threads | ||
| 27 | 2026.08.17 23:52:06.010373 [ 423933 ] {} <Information> BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 28 | 2026.08.17 23:52:06.011062 [ 423933 ] {} <Information> BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 29 | 2026.08.17 23:52:06.012229 [ 423933 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. | ||
| 30 | 2026.08.17 23:52:06.012245 [ 423933 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 31 | 2026.08.17 23:52:06.012386 [ 423933 ] {} <Information> Application: Listening for replica communication (interserver): http://[::1]:19401 | ||
| 32 | 2026.08.17 23:52:06.013376 [ 423933 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' | ||
| 33 | 2026.08.17 23:52:06.014888 [ 423933 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 34 | 2026.08.17 23:52:06.015385 [ 423933 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 35 | 2026.08.17 23:52:06.015579 [ 423933 ] {} <Debug> Access(user directories): Added users_xml access storage 'users_xml', path: /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml | ||
| 36 | 2026.08.17 23:52:06.017384 [ 423933 ] {} <Information> Context: Initialized background executor for merges and mutations with num_threads=16, num_tasks=32, scheduling_policy=round_robin | ||
| 37 | 2026.08.17 23:52:06.017598 [ 423933 ] {} <Information> Context: Initialized background executor for move operations with num_threads=8, num_tasks=8 | ||
| 38 | 2026.08.17 23:52:06.017780 [ 423933 ] {} <Information> Context: Initialized background executor for fetches with num_threads=8, num_tasks=8 | ||
| 39 | 2026.08.17 23:52:06.018963 [ 423933 ] {} <Information> Context: Initialized background executor for common operations (e.g. clearing old parts) with num_threads=8, num_tasks=8 | ||
| 40 | 2026.08.17 23:52:06.019547 [ 423933 ] {} <Information> DNSCacheUpdater: Update period 15 seconds | ||
| 41 | 2026.08.17 23:52:06.019553 [ 423933 ] {} <Information> Application: Loading metadata from /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/data/ | ||
| 42 | 2026.08.17 23:52:06.019570 [ 424510 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 43 | 2026.08.17 23:52:06.019636 [ 424510 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 44 | 2026.08.17 23:52:06.033943 [ 423933 ] {} <Information> DatabaseAtomic (system): Metadata processed, database system has 0 tables and 0 dictionaries in total. | ||
| 45 | 2026.08.17 23:52:06.033952 [ 423933 ] {} <Information> TablesLoader: Parsed metadata of 0 tables in 1 databases in 4.969e-05 sec | ||
| 46 | 2026.08.17 23:52:06.033957 [ 423933 ] {} <Trace> ReferentialDeps: No tables | ||
| 47 | 2026.08.17 23:52:06.033959 [ 423933 ] {} <Trace> LoadingDeps: No tables | ||
| 48 | 2026.08.17 23:52:06.033978 [ 423933 ] {} <Information> DatabaseAtomic (system): Starting up tables. | ||
| 49 | 2026.08.17 23:52:06.033984 [ 423933 ] {} <Debug> SystemLog: Not creating system.query_log since corresponding section 'query_log' is missing from config | ||
| 50 | 2026.08.17 23:52:06.033988 [ 423933 ] {} <Debug> SystemLog: Not creating system.query_thread_log since corresponding section 'query_thread_log' is missing from config | ||
| 51 | 2026.08.17 23:52:06.033991 [ 423933 ] {} <Debug> SystemLog: Not creating system.part_log since corresponding section 'part_log' is missing from config | ||
| 52 | 2026.08.17 23:52:06.033994 [ 423933 ] {} <Debug> SystemLog: Not creating system.trace_log since corresponding section 'trace_log' is missing from config | ||
| 53 | 2026.08.17 23:52:06.033997 [ 423933 ] {} <Debug> SystemLog: Not creating system.crash_log since corresponding section 'crash_log' is missing from config | ||
| 54 | 2026.08.17 23:52:06.034000 [ 423933 ] {} <Debug> SystemLog: Not creating system.text_log since corresponding section 'text_log' is missing from config | ||
| 55 | 2026.08.17 23:52:06.034004 [ 423933 ] {} <Debug> SystemLog: Creating system.metric_log from metric_log | ||
| 56 | 2026.08.17 23:52:06.034947 [ 423933 ] {} <Debug> SystemLog: Not creating system.filesystem_cache_log since corresponding section 'filesystem_cache_log' is missing from config | ||
| 57 | 2026.08.17 23:52:06.034957 [ 423933 ] {} <Debug> SystemLog: Not creating system.filesystem_read_prefetches_log since corresponding section 'filesystem_read_prefetches_log' is missing from config | ||
| 58 | 2026.08.17 23:52:06.034961 [ 423933 ] {} <Debug> SystemLog: Creating system.asynchronous_metric_log from asynchronous_metric_log | ||
| 59 | 2026.08.17 23:52:06.035032 [ 423933 ] {} <Debug> SystemLog: Creating system.opentelemetry_span_log from opentelemetry_span_log | ||
| 60 | 2026.08.17 23:52:06.035120 [ 423933 ] {} <Debug> SystemLog: Not creating system.query_views_log since corresponding section 'query_views_log' is missing from config | ||
| 61 | 2026.08.17 23:52:06.035123 [ 423933 ] {} <Debug> SystemLog: Not creating system.zookeeper_log since corresponding section 'zookeeper_log' is missing from config | ||
| 62 | 2026.08.17 23:52:06.035126 [ 423933 ] {} <Debug> SystemLog: Not creating system.session_log since corresponding section 'session_log' is missing from config | ||
| 63 | 2026.08.17 23:52:06.035129 [ 423933 ] {} <Debug> SystemLog: Not creating system.transactions_info_log since corresponding section 'transactions_info_log' is missing from config | ||
| 64 | 2026.08.17 23:52:06.035144 [ 423933 ] {} <Debug> SystemLog: Not creating system.processors_profile_log since corresponding section 'processors_profile_log' is missing from config | ||
| 65 | 2026.08.17 23:52:06.035147 [ 423933 ] {} <Debug> SystemLog: Not creating system.asynchronous_insert_log since corresponding section 'asynchronous_insert_log' is missing from config | ||
| 66 | 2026.08.17 23:52:06.041115 [ 423933 ] {} <Information> DatabaseCatalog: Found 0 partially dropped tables. Will load them and retry removal. | ||
| 67 | 2026.08.17 23:52:06.045825 [ 423933 ] {} <Information> DatabaseAtomic (default): Metadata processed, database default has 0 tables and 0 dictionaries in total. | ||
| 68 | 2026.08.17 23:52:06.045834 [ 423933 ] {} <Information> TablesLoader: Parsed metadata of 0 tables in 1 databases in 2.866e-05 sec | ||
| 69 | 2026.08.17 23:52:06.045837 [ 423933 ] {} <Trace> ReferentialDeps: No tables | ||
| 70 | 2026.08.17 23:52:06.045839 [ 423933 ] {} <Trace> LoadingDeps: No tables | ||
| 71 | 2026.08.17 23:52:06.045850 [ 423933 ] {} <Information> DatabaseAtomic (default): Starting up tables. | ||
| 72 | 2026.08.17 23:52:06.045899 [ 423933 ] {} <Information> UserDefinedSQLObjectsLoaderFromDisk: Loading user defined objects from /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/data/user_defined/ | ||
| 73 | 2026.08.17 23:52:06.045910 [ 423933 ] {} <Debug> UserDefinedSQLObjectsLoaderFromDisk: User defined objects loaded | ||
| 74 | 2026.08.17 23:52:06.045912 [ 423933 ] {} <Debug> Application: Loaded metadata. | ||
| 75 | 2026.08.17 23:52:06.045924 [ 423933 ] {} <Information> Application: Tasks stats provider: procfs | ||
| 76 | 2026.08.17 23:52:06.045926 [ 423933 ] {} <Information> Application: It looks like the process has no CAP_SYS_NICE capability, the setting 'os_thread_priority' will have no effect. It could happen due to incorrect ClickHouse package installation. You could resolve the problem manually with 'sudo setcap cap_sys_nice=+ep /work/oxidecomputer/omicron/out/clickhouse/clickhouse'. Note that it will not work on 'nosuid' mounted filesystems. | ||
| 77 | 2026.08.17 23:52:06.045998 [ 423933 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. | ||
| 78 | 2026.08.17 23:52:06.046392 [ 423933 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 12.58 MiB, peak 13.26 MiB, free memory in arenas 9.73 MiB, will set to 372.06 MiB (RSS), difference: 359.48 MiB | ||
| 79 | 2026.08.17 23:52:06.046689 [ 423933 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 6.92e-06 sec. | ||
| 80 | 2026.08.17 23:52:06.046937 [ 423933 ] {} <Information> Application: Listening for http://[::1]:19301 | ||
| 81 | 2026.08.17 23:52:06.046958 [ 425097 ] {} <Debug> DDLWorker: Starting DDLWorker thread | ||
| 82 | 2026.08.17 23:52:06.048615 [ 425097 ] {} <Debug> DDLWorker: Initializing DDLWorker thread | ||
| 83 | 2026.08.17 23:52:06.048619 [ 423933 ] {} <Information> Application: Listening for native protocol (tcp): [::1]:19201 | ||
| 84 | 2026.08.17 23:52:06.046969 [ 425098 ] {} <Debug> DDLWorker: Started DDLWorker cleanup thread | ||
| 85 | 2026.08.17 23:52:06.049136 [ 423933 ] {} <Information> Application: Ready for connections. | ||
| 86 | 2026.08.17 23:52:06.049290 [ 425097 ] {} <Error> virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 87 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 88 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 89 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 90 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 91 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 92 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 93 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 94 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 95 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 96 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 97 | |||
| 98 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 99 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 100 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 101 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 102 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 103 | 5. zkutil::ZooKeeper::ZooKeeper(Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f66f9 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 104 | 6. DB::Context::getZooKeeper() const @ 0x000000001166afca in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 105 | 7. DB::DDLWorker::getAndSetZooKeeper() @ 0x00000000116ec1ed in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 106 | 8. DB::DDLWorker::initializeMainThread() @ 0x00000000116fc81b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 107 | 9. DB::DDLWorker::runMainThread() @ 0x00000000116e7280 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 108 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000116fd92c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 109 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 110 | 12. ? @ 0x00007fd83ba48a83 in ? | ||
| 111 | 13. ? @ 0x00007fd83bada890 in ? | ||
| 112 | (version 23.8.7.24 (official build)) | ||
| 113 | 2026.08.17 23:52:06.049813 [ 424014 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56566 | ||
| 114 | 2026.08.17 23:52:06.050454 [ 424014 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 115 | 2026.08.17 23:52:06.050467 [ 424014 ] {} <Debug> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Authenticating user 'default' from [::1]:56566 | ||
| 116 | 2026.08.17 23:52:06.050478 [ 424014 ] {} <Debug> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 117 | 2026.08.17 23:52:06.050481 [ 424014 ] {} <Debug> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 118 | 2026.08.17 23:52:06.050557 [ 424014 ] {} <Trace> ContextAccess (default): Settings: readonly = 0, allow_ddl = true, allow_introspection_functions = false | ||
| 119 | 2026.08.17 23:52:06.050566 [ 424014 ] {} <Trace> ContextAccess (default): List of all grants: GRANT SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* | ||
| 120 | 2026.08.17 23:52:06.050570 [ 424014 ] {} <Trace> ContextAccess (default): List of all grants including implicit: GRANT SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* | ||
| 121 | 2026.08.17 23:52:06.050909 [ 424016 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56572 | ||
| 122 | 2026.08.17 23:52:06.050921 [ 424015 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56570 | ||
| 123 | 2026.08.17 23:52:06.051063 [ 424015 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 124 | 2026.08.17 23:52:06.051074 [ 424015 ] {} <Debug> TCP-Session: e444f30b-9559-49da-a89d-921a866903e7 Authenticating user 'default' from [::1]:56570 | ||
| 125 | 2026.08.17 23:52:06.051082 [ 424015 ] {} <Debug> TCP-Session: e444f30b-9559-49da-a89d-921a866903e7 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 126 | 2026.08.17 23:52:06.051072 [ 424016 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 127 | 2026.08.17 23:52:06.051084 [ 424015 ] {} <Debug> TCP-Session: e444f30b-9559-49da-a89d-921a866903e7 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 128 | 2026.08.17 23:52:06.051094 [ 424016 ] {} <Debug> TCP-Session: 31d2804f-7a6a-4904-a91f-9b4e4cd6922d Authenticating user 'default' from [::1]:56572 | ||
| 129 | 2026.08.17 23:52:06.051099 [ 424016 ] {} <Debug> TCP-Session: 31d2804f-7a6a-4904-a91f-9b4e4cd6922d Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 130 | 2026.08.17 23:52:06.051101 [ 424016 ] {} <Debug> TCP-Session: 31d2804f-7a6a-4904-a91f-9b4e4cd6922d Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 131 | 2026.08.17 23:52:06.051239 [ 425106 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56584 | ||
| 132 | 2026.08.17 23:52:06.051269 [ 424015 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 133 | 2026.08.17 23:52:06.051268 [ 425102 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56576 | ||
| 134 | 2026.08.17 23:52:06.051287 [ 424015 ] {} <Debug> TCP-Session: e444f30b-9559-49da-a89d-921a866903e7 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 135 | 2026.08.17 23:52:06.051294 [ 425103 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56578 | ||
| 136 | 2026.08.17 23:52:06.051297 [ 425101 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56574 | ||
| 137 | 2026.08.17 23:52:06.051306 [ 425104 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56580 | ||
| 138 | 2026.08.17 23:52:06.051394 [ 425102 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 139 | 2026.08.17 23:52:06.051388 [ 425105 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56582 | ||
| 140 | 2026.08.17 23:52:06.051414 [ 425106 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 141 | 2026.08.17 23:52:06.051432 [ 425103 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 142 | 2026.08.17 23:52:06.051434 [ 425101 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 143 | 2026.08.17 23:52:06.052680 [ 425103 ] {} <Debug> TCP-Session: ad1ac43b-f78f-4718-b2c5-f276d7dc672b Authenticating user 'default' from [::1]:56578 | ||
| 144 | 2026.08.17 23:52:06.051439 [ 425104 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 145 | 2026.08.17 23:52:06.052644 [ 425102 ] {} <Debug> TCP-Session: 040a616f-1d20-40f3-aad4-12c3260ca8e8 Authenticating user 'default' from [::1]:56576 | ||
| 146 | 2026.08.17 23:52:06.052681 [ 425106 ] {} <Debug> TCP-Session: 910b1cca-0222-422d-8ab2-4e3745e63cbc Authenticating user 'default' from [::1]:56584 | ||
| 147 | 2026.08.17 23:52:06.052683 [ 425101 ] {} <Debug> TCP-Session: b8285f84-965e-4130-87b2-b699d711ba77 Authenticating user 'default' from [::1]:56574 | ||
| 148 | 2026.08.17 23:52:06.052694 [ 425105 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 149 | 2026.08.17 23:52:06.052697 [ 425101 ] {} <Debug> TCP-Session: b8285f84-965e-4130-87b2-b699d711ba77 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 150 | 2026.08.17 23:52:06.052698 [ 425106 ] {} <Debug> TCP-Session: 910b1cca-0222-422d-8ab2-4e3745e63cbc Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 151 | 2026.08.17 23:52:06.052701 [ 425106 ] {} <Debug> TCP-Session: 910b1cca-0222-422d-8ab2-4e3745e63cbc Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 152 | 2026.08.17 23:52:06.052701 [ 425101 ] {} <Debug> TCP-Session: b8285f84-965e-4130-87b2-b699d711ba77 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 153 | 2026.08.17 23:52:06.052705 [ 425105 ] {} <Debug> TCP-Session: 6411da06-f88c-4e2b-bf3a-efa646da50f0 Authenticating user 'default' from [::1]:56582 | ||
| 154 | 2026.08.17 23:52:06.052712 [ 425105 ] {} <Debug> TCP-Session: 6411da06-f88c-4e2b-bf3a-efa646da50f0 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 155 | 2026.08.17 23:52:06.052716 [ 425105 ] {} <Debug> TCP-Session: 6411da06-f88c-4e2b-bf3a-efa646da50f0 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 156 | 2026.08.17 23:52:06.052689 [ 425103 ] {} <Debug> TCP-Session: ad1ac43b-f78f-4718-b2c5-f276d7dc672b Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 157 | 2026.08.17 23:52:06.052722 [ 425103 ] {} <Debug> TCP-Session: ad1ac43b-f78f-4718-b2c5-f276d7dc672b Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 158 | 2026.08.17 23:52:06.052715 [ 425104 ] {} <Debug> TCP-Session: 7fb269c2-482e-4967-b71c-3170be13d6a3 Authenticating user 'default' from [::1]:56580 | ||
| 159 | 2026.08.17 23:52:06.052695 [ 425102 ] {} <Debug> TCP-Session: 040a616f-1d20-40f3-aad4-12c3260ca8e8 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 160 | 2026.08.17 23:52:06.052733 [ 425104 ] {} <Debug> TCP-Session: 7fb269c2-482e-4967-b71c-3170be13d6a3 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 161 | 2026.08.17 23:52:06.052733 [ 425102 ] {} <Debug> TCP-Session: 040a616f-1d20-40f3-aad4-12c3260ca8e8 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 162 | 2026.08.17 23:52:06.052739 [ 425104 ] {} <Debug> TCP-Session: 7fb269c2-482e-4967-b71c-3170be13d6a3 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 163 | 2026.08.17 23:52:07.000501 [ 425094 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 372.14 MiB, peak 374.13 MiB, free memory in arenas 8.33 MiB, will set to 375.29 MiB (RSS), difference: 3.15 MiB | ||
| 164 | 2026.08.17 23:52:07.128849 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 165 | 2026.08.17 23:52:07.129036 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Trace> executeQuery: Query span trace_id for opentelemetry log: 72382db7-1ff4-3af4-9280-ff69ce065edc | ||
| 166 | 2026.08.17 23:52:07.129076 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Debug> executeQuery: (from [::1]:56566) /* We purposefully split our DB schema into two *disjoint* files: | ||
| 167 | * `db-init-1.sql` and `db-init-2.sql`. The purpose of this split is to shorten | ||
| 168 | * the duration of our replicated tests. These tests only use a subset of the | ||
| 169 | * tables defined in the full schema. We put the tables used by the replicated | ||
| 170 | * tests in `db-init-1.sql`, and the remainder of the tables in `db-init-2.sql`. | ||
| 171 | * This minimizes test time by reducing the cost to load a schema. In | ||
| 172 | * production, we load `db-init-1.sql` followed by `db-init-2.sql` so we have | ||
| 173 | * the full schema. If we end up needing to use more tables in replicated tests | ||
| 174 | * we can go ahead and move them into `db-init-1.sql`, removing them from | ||
| 175 | * `db-init-2.sql`. Conversely, if we stop using given tables in our tests we | ||
| 176 | * can move them from `db-init-1.sql` into `db-init-2.sql` and keep our test | ||
| 177 | * times minimal. | ||
| 178 | |||
| 179 | * The reason to keep the two files disjoint is so that we don't have to | ||
| 180 | * maintain consistency between table definitions. All tables are defined | ||
| 181 | * once. However, in order to write a test that ensures the tables are in fact | ||
| 182 | * disjoint, we must create the `oximeter` database in both files so we can load | ||
| 183 | * them in isolation. | ||
| 184 | */ | ||
| 185 | CREATE DATABASE IF NOT EXISTS oximeter ON CLUSTER oximeter_cluster (stage: Complete) | ||
| 186 | 2026.08.17 23:52:07.129142 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Trace> ContextAccess (default): Access granted: CREATE DATABASE ON oximeter.* | ||
| 187 | 2026.08.17 23:52:07.132941 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Information> ZooKeeperClient: Connected to ZooKeeper at [::1]:19001 with session_id 4 | ||
| 188 | 2026.08.17 23:52:07.133178 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Information> ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled | ||
| 189 | 2026.08.17 23:52:07.133181 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Information> ZooKeeperClient: Keeper feature flag MULTI_READ: enabled | ||
| 190 | 2026.08.17 23:52:07.133182 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Information> ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: disabled | ||
| 191 | 2026.08.17 23:52:07.133186 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Trace> ZooKeeper: Initialized, hosts: [::1]:19001,[::1]:19002,[::1]:19003 | ||
| 192 | 2026.08.17 23:52:08.000708 [ 425094 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 379.36 MiB, peak 379.36 MiB, free memory in arenas 8.33 MiB, will set to 375.55 MiB (RSS), difference: -3.81 MiB | ||
| 193 | 2026.08.17 23:52:11.055754 [ 425097 ] {} <Debug> DDLWorker: Initialized DDLWorker thread | ||
| 194 | 2026.08.17 23:52:11.056813 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 195 | 2026.08.17 23:52:11.056817 [ 425097 ] {} <Trace> DDLWorker: Don't have unfinished tasks after restarting | ||
| 196 | 2026.08.17 23:52:11.056818 [ 425098 ] {} <Debug> DDLWorker: Cleaning queue | ||
| 197 | 2026.08.17 23:52:11.056899 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=1, queue_size=1, entries=query-0000000000..query-0000000000, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=none | ||
| 198 | 2026.08.17 23:52:11.056903 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000000 | ||
| 199 | 2026.08.17 23:52:11.056906 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000000 | ||
| 200 | 2026.08.17 23:52:11.057122 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000000 (CREATE DATABASE IF NOT EXISTS oximeter UUID 'c0a94260-6234-449c-8c74-f5c4edf4ee5c') | ||
| 201 | 2026.08.17 23:52:11.062830 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE DATABASE IF NOT EXISTS oximeter UUID 'c0a94260-6234-449c-8c74-f5c4edf4ee5c' | ||
| 202 | 2026.08.17 23:52:11.062888 [ 425097 ] {ba958a36-df77-476f-b5bd-499c038a8c2f} <Trace> executeQuery: Query span trace_id for opentelemetry log: 72382db7-1ff4-3af4-9280-ff69ce065edc | ||
| 203 | 2026.08.17 23:52:11.062912 [ 425097 ] {ba958a36-df77-476f-b5bd-499c038a8c2f} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: a280c8a5-755d-479b-8666-3ff95c6b5df2) /* ddl_entry=query-0000000000 */ CREATE DATABASE IF NOT EXISTS oximeter UUID 'c0a94260-6234-449c-8c74-f5c4edf4ee5c' (stage: Complete) | ||
| 204 | 2026.08.17 23:52:11.070007 [ 425097 ] {ba958a36-df77-476f-b5bd-499c038a8c2f} <Information> DatabaseAtomic (oximeter): Metadata processed, database oximeter has 0 tables and 0 dictionaries in total. | ||
| 205 | 2026.08.17 23:52:11.070016 [ 425097 ] {ba958a36-df77-476f-b5bd-499c038a8c2f} <Information> TablesLoader: Parsed metadata of 0 tables in 1 databases in 4.476e-05 sec | ||
| 206 | 2026.08.17 23:52:11.070020 [ 425097 ] {ba958a36-df77-476f-b5bd-499c038a8c2f} <Trace> ReferentialDeps: No tables | ||
| 207 | 2026.08.17 23:52:11.070023 [ 425097 ] {ba958a36-df77-476f-b5bd-499c038a8c2f} <Trace> LoadingDeps: No tables | ||
| 208 | 2026.08.17 23:52:11.070037 [ 425097 ] {ba958a36-df77-476f-b5bd-499c038a8c2f} <Information> DatabaseAtomic (oximeter): Starting up tables. | ||
| 209 | 2026.08.17 23:52:11.070089 [ 425097 ] {ba958a36-df77-476f-b5bd-499c038a8c2f} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000000 */ CREATE DATABASE IF NOT EXISTS oximeter UUID 'c0a94260-6234-449c-8c74-f5c4edf4ee5c' | ||
| 210 | 2026.08.17 23:52:11.076020 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 211 | 2026.08.17 23:52:11.668393 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Debug> executeQuery: Read 2 rows, 94.00 B in 4.539338 sec., 0.44059287940223885 rows/sec., 20.71 B/sec. | ||
| 212 | 2026.08.17 23:52:11.668428 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Debug> MemoryTracker: Peak memory usage (for query): 4.11 MiB. | ||
| 213 | 2026.08.17 23:52:11.668432 [ 424014 ] {a280c8a5-755d-479b-8666-3ff95c6b5df2} <Debug> TCPHandler: Processed in 4.539658557 sec. | ||
| 214 | 2026.08.17 23:52:11.668947 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 215 | 2026.08.17 23:52:11.669076 [ 424014 ] {c15a2d35-8b50-4b64-aa19-7f271e0a411f} <Trace> executeQuery: Query span trace_id for opentelemetry log: 204ea5ef-62f4-db6b-c243-a5166fd03c0c | ||
| 216 | 2026.08.17 23:52:11.669204 [ 424014 ] {c15a2d35-8b50-4b64-aa19-7f271e0a411f} <Debug> executeQuery: (from [::1]:56566) /* The version table contains metadata about the `oximeter` database */ | ||
| 217 | CREATE TABLE IF NOT EXISTS oximeter.version ON CLUSTER oximeter_cluster ( value UInt64, timestamp DateTime64(9, 'UTC') ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) (stage: Complete) | ||
| 218 | 2026.08.17 23:52:11.669247 [ 424014 ] {c15a2d35-8b50-4b64-aa19-7f271e0a411f} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.version | ||
| 219 | 2026.08.17 23:52:11.669311 [ 424014 ] {c15a2d35-8b50-4b64-aa19-7f271e0a411f} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.version | ||
| 220 | 2026.08.17 23:52:11.677882 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 221 | 2026.08.17 23:52:11.677889 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 222 | 2026.08.17 23:52:11.677954 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=2, queue_size=2, entries=query-0000000000..query-0000000001, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000000, last_skipped_entry_name=none | ||
| 223 | 2026.08.17 23:52:11.677963 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000001 | ||
| 224 | 2026.08.17 23:52:11.677966 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000001 | ||
| 225 | 2026.08.17 23:52:11.683399 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000001 (CREATE TABLE IF NOT EXISTS oximeter.version UUID '157c979f-eec8-4d9d-8340-ef2e89475af4' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp)) | ||
| 226 | 2026.08.17 23:52:11.692910 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.version UUID '157c979f-eec8-4d9d-8340-ef2e89475af4' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) | ||
| 227 | 2026.08.17 23:52:11.692976 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Trace> executeQuery: Query span trace_id for opentelemetry log: 204ea5ef-62f4-db6b-c243-a5166fd03c0c | ||
| 228 | 2026.08.17 23:52:11.693065 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: c15a2d35-8b50-4b64-aa19-7f271e0a411f) /* ddl_entry=query-0000000001 */ CREATE TABLE IF NOT EXISTS oximeter.version UUID '157c979f-eec8-4d9d-8340-ef2e89475af4' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) (stage: Complete) | ||
| 229 | 2026.08.17 23:52:11.693633 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Trace> StoragePolicy (default): Storage policy default created, total volumes 1 | ||
| 230 | 2026.08.17 23:52:11.699843 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Loading data parts | ||
| 231 | 2026.08.17 23:52:11.700099 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): There are no data parts | ||
| 232 | 2026.08.17 23:52:11.727286 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Creating table /clickhouse/tables/1/version | ||
| 233 | 2026.08.17 23:52:11.738060 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Information> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): It looks like the table /clickhouse/tables/1/version was created by another server at the same moment, will retry | ||
| 234 | 2026.08.17 23:52:11.738225 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): This table /clickhouse/tables/1/version is already created, will add new replica | ||
| 235 | 2026.08.17 23:52:11.738526 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Creating replica /clickhouse/tables/1/version/replicas/1 | ||
| 236 | 2026.08.17 23:52:11.778878 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Creating shared ID for table oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4) | ||
| 237 | 2026.08.17 23:52:11.780564 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Initializing table shared ID with 157c979f-eec8-4d9d-8340-ef2e89475af4 | ||
| 238 | 2026.08.17 23:52:11.786786 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Starting up table | ||
| 239 | 2026.08.17 23:52:11.794346 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Information> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Became leader | ||
| 240 | 2026.08.17 23:52:11.794356 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Trace> oximeter.version (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 241 | 2026.08.17 23:52:11.794396 [ 424534 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 242 | 2026.08.17 23:52:11.794408 [ 424534 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 243 | 2026.08.17 23:52:11.800868 [ 424534 ] {} <Information> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Replica 2 has log pointer '', approximate 1 queue lag and 0 queue size | ||
| 244 | 2026.08.17 23:52:11.800875 [ 424534 ] {} <Information> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Will mimic 2 | ||
| 245 | 2026.08.17 23:52:11.808340 [ 424534 ] {} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Queued 0 parts to be fetched, 0 parts ignored | ||
| 246 | 2026.08.17 23:52:11.809194 [ 424534 ] {} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Copied 0 queue entries, 0 entries ignored | ||
| 247 | 2026.08.17 23:52:11.809251 [ 424534 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Parts in ZooKeeper after mimic: | ||
| 248 | 2026.08.17 23:52:11.809258 [ 424534 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Enqueued fetches after mimic: | ||
| 249 | 2026.08.17 23:52:11.812717 [ 424534 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 250 | 2026.08.17 23:52:11.812769 [ 424534 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Queue initialized | ||
| 251 | 2026.08.17 23:52:11.812772 [ 424534 ] {} <Debug> oximeter.version (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/version/replicas/1/queue | ||
| 252 | 2026.08.17 23:52:11.812811 [ 424534 ] {} <Debug> oximeter.version (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 253 | 2026.08.17 23:52:11.812858 [ 424534 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Loaded queue | ||
| 254 | 2026.08.17 23:52:11.819850 [ 424521 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 6134ms | ||
| 255 | 2026.08.17 23:52:11.819910 [ 424534 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 256 | 2026.08.17 23:52:11.820003 [ 425097 ] {dec5e2c8-5678-4ec7-b6b8-03b04f0785b1} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000001 */ CREATE TABLE IF NOT EXISTS oximeter.version UUID '157c979f-eec8-4d9d-8340-ef2e89475af4' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) | ||
| 257 | 2026.08.17 23:52:11.830471 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 258 | 2026.08.17 23:52:11.944810 [ 424014 ] {c15a2d35-8b50-4b64-aa19-7f271e0a411f} <Debug> executeQuery: Read 2 rows, 94.00 B in 0.275709 sec., 7.254025077164692 rows/sec., 340.94 B/sec. | ||
| 259 | 2026.08.17 23:52:11.944837 [ 424014 ] {c15a2d35-8b50-4b64-aa19-7f271e0a411f} <Debug> TCPHandler: Processed in 0.276068778 sec. | ||
| 260 | 2026.08.17 23:52:11.945304 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 261 | 2026.08.17 23:52:11.945370 [ 424014 ] {5a817b06-0a26-4618-b28b-2d3e91b76496} <Trace> executeQuery: Query span trace_id for opentelemetry log: 8180f263-a2c5-6d72-6d5e-e874f0144e75 | ||
| 262 | 2026.08.17 23:52:11.945495 [ 424014 ] {5a817b06-0a26-4618-b28b-2d3e91b76496} <Debug> executeQuery: (from [::1]:56566) /* The measurement tables contain all individual samples from each timeseries. | ||
| 263 | * | ||
| 264 | * Each table stores a single datum type, and otherwise contains nearly the same | ||
| 265 | * structure. The primary sorting key is on the timeseries name, key, and then | ||
| 266 | * timestamp, so that all timeseries from the same schema are grouped, followed | ||
| 267 | * by all samples from the same timeseries. | ||
| 268 | * | ||
| 269 | * This reflects that one usually looks up the _key_ in one or more field table, | ||
| 270 | * and then uses that to index quickly into the measurements tables. | ||
| 271 | */ | ||
| 272 | CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local ON CLUSTER oximeter_cluster ( timeseries_name String, timeseries_key UInt64, start_time DateTime64(9, 'UTC'), timestamp DateTime64(9, 'UTC'), datum Nullable(Float64) ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + INTERVAL 30 DAY (stage: Complete) | ||
| 273 | 2026.08.17 23:52:11.945537 [ 424014 ] {5a817b06-0a26-4618-b28b-2d3e91b76496} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.measurements_cumulativef64_local | ||
| 274 | 2026.08.17 23:52:11.945624 [ 424014 ] {5a817b06-0a26-4618-b28b-2d3e91b76496} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.measurements_cumulativef64_local | ||
| 275 | 2026.08.17 23:52:11.968794 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 276 | 2026.08.17 23:52:11.968805 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 277 | 2026.08.17 23:52:11.977189 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=3, queue_size=3, entries=query-0000000000..query-0000000002, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000001, last_skipped_entry_name=none | ||
| 278 | 2026.08.17 23:52:11.977207 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000002 | ||
| 279 | 2026.08.17 23:52:11.977210 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000002 | ||
| 280 | 2026.08.17 23:52:11.981724 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000002 (CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID '24dea9d7-064d-49c2-9599-9587402bf78f' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30)) | ||
| 281 | 2026.08.17 23:52:11.990025 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID '24dea9d7-064d-49c2-9599-9587402bf78f' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30) | ||
| 282 | 2026.08.17 23:52:11.990095 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Trace> executeQuery: Query span trace_id for opentelemetry log: 8180f263-a2c5-6d72-6d5e-e874f0144e75 | ||
| 283 | 2026.08.17 23:52:11.990228 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 5a817b06-0a26-4618-b28b-2d3e91b76496) /* ddl_entry=query-0000000002 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID '24dea9d7-064d-49c2-9599-9587402bf78f' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30) (stage: Complete) | ||
| 284 | 2026.08.17 23:52:11.999805 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Loading data parts | ||
| 285 | 2026.08.17 23:52:12.001305 [ 425094 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 375.73 MiB, peak 379.36 MiB, free memory in arenas 8.33 MiB, will set to 377.77 MiB (RSS), difference: 2.04 MiB | ||
| 286 | 2026.08.17 23:52:12.001626 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): There are no data parts | ||
| 287 | 2026.08.17 23:52:12.006727 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Creating table /clickhouse/tables/1/measurements_cumulativef64_local | ||
| 288 | 2026.08.17 23:52:12.031951 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Creating shared ID for table oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) | ||
| 289 | 2026.08.17 23:52:12.031981 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Shared ID for table oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) doesn't exist in ZooKeeper on path /clickhouse/tables/1/measurements_cumulativef64_local/table_shared_id | ||
| 290 | 2026.08.17 23:52:12.031984 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Got candidate ID 24dea9d7-064d-49c2-9599-9587402bf78f, will try to create it in ZooKeeper on path /clickhouse/tables/1/measurements_cumulativef64_local/table_shared_id | ||
| 291 | 2026.08.17 23:52:12.044757 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Initializing table shared ID with 24dea9d7-064d-49c2-9599-9587402bf78f | ||
| 292 | 2026.08.17 23:52:12.051661 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Starting up table | ||
| 293 | 2026.08.17 23:52:12.061762 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Information> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Became leader | ||
| 294 | 2026.08.17 23:52:12.061772 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 295 | 2026.08.17 23:52:12.061815 [ 424570 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 296 | 2026.08.17 23:52:12.061847 [ 424570 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 297 | 2026.08.17 23:52:12.068839 [ 424570 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 298 | 2026.08.17 23:52:12.068919 [ 424570 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 299 | 2026.08.17 23:52:12.068924 [ 424570 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/measurements_cumulativef64_local/replicas/1/queue | ||
| 300 | 2026.08.17 23:52:12.069548 [ 424570 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 301 | 2026.08.17 23:52:12.069615 [ 424570 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 302 | 2026.08.17 23:52:12.078591 [ 424570 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 303 | 2026.08.17 23:52:12.078678 [ 424601 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 6356ms | ||
| 304 | 2026.08.17 23:52:12.078760 [ 425097 ] {7e500458-7a32-40e2-89be-88bc3c8c7d1f} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000002 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID '24dea9d7-064d-49c2-9599-9587402bf78f' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30) | ||
| 305 | 2026.08.17 23:52:12.087337 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 306 | 2026.08.17 23:52:12.238449 [ 424014 ] {5a817b06-0a26-4618-b28b-2d3e91b76496} <Debug> executeQuery: Read 2 rows, 94.00 B in 0.293061 sec., 6.824517762513606 rows/sec., 320.75 B/sec. | ||
| 307 | 2026.08.17 23:52:12.238490 [ 424014 ] {5a817b06-0a26-4618-b28b-2d3e91b76496} <Debug> TCPHandler: Processed in 0.29326464 sec. | ||
| 308 | 2026.08.17 23:52:12.238928 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 309 | 2026.08.17 23:52:12.238990 [ 424014 ] {4f14b02b-fd28-4a32-a989-a07041e49557} <Trace> executeQuery: Query span trace_id for opentelemetry log: 3cfe1541-76c2-719b-bb47-1d1a651a6671 | ||
| 310 | 2026.08.17 23:52:12.239086 [ 424014 ] {4f14b02b-fd28-4a32-a989-a07041e49557} <Debug> executeQuery: (from [::1]:56566) CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 ON CLUSTER oximeter_cluster AS oximeter.measurements_cumulativef64_local ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 311 | 2026.08.17 23:52:12.240421 [ 424014 ] {4f14b02b-fd28-4a32-a989-a07041e49557} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.measurements_cumulativef64 | ||
| 312 | 2026.08.17 23:52:12.240430 [ 424014 ] {4f14b02b-fd28-4a32-a989-a07041e49557} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 313 | 2026.08.17 23:52:12.240436 [ 424014 ] {4f14b02b-fd28-4a32-a989-a07041e49557} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 314 | 2026.08.17 23:52:12.240523 [ 424014 ] {4f14b02b-fd28-4a32-a989-a07041e49557} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.measurements_cumulativef64 | ||
| 315 | 2026.08.17 23:52:12.240525 [ 424014 ] {4f14b02b-fd28-4a32-a989-a07041e49557} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 316 | 2026.08.17 23:52:12.248950 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 317 | 2026.08.17 23:52:12.248957 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 318 | 2026.08.17 23:52:12.254581 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=4, queue_size=4, entries=query-0000000000..query-0000000003, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000002, last_skipped_entry_name=none | ||
| 319 | 2026.08.17 23:52:12.254587 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000003 | ||
| 320 | 2026.08.17 23:52:12.254590 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000003 | ||
| 321 | 2026.08.17 23:52:12.254912 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000003 (CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '26440754-ec6a-43fc-b101-925af35ef3c9' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 322 | 2026.08.17 23:52:12.257973 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '26440754-ec6a-43fc-b101-925af35ef3c9' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 323 | 2026.08.17 23:52:12.258024 [ 425097 ] {e47d6b59-ef9e-4817-8254-89b514d496fc} <Trace> executeQuery: Query span trace_id for opentelemetry log: 3cfe1541-76c2-719b-bb47-1d1a651a6671 | ||
| 324 | 2026.08.17 23:52:12.258099 [ 425097 ] {e47d6b59-ef9e-4817-8254-89b514d496fc} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 4f14b02b-fd28-4a32-a989-a07041e49557) /* ddl_entry=query-0000000003 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '26440754-ec6a-43fc-b101-925af35ef3c9' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 325 | 2026.08.17 23:52:12.258536 [ 425097 ] {e47d6b59-ef9e-4817-8254-89b514d496fc} <Debug> StorageDistributed (measurements_cumulativef64): Auto-increment is 0 | ||
| 326 | 2026.08.17 23:52:12.265059 [ 425097 ] {e47d6b59-ef9e-4817-8254-89b514d496fc} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000003 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '26440754-ec6a-43fc-b101-925af35ef3c9' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 327 | 2026.08.17 23:52:12.268281 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 328 | 2026.08.17 23:52:12.355280 [ 424014 ] {4f14b02b-fd28-4a32-a989-a07041e49557} <Debug> executeQuery: Read 2 rows, 94.00 B in 0.116274 sec., 17.200749952697937 rows/sec., 808.44 B/sec. | ||
| 329 | 2026.08.17 23:52:12.355304 [ 424014 ] {4f14b02b-fd28-4a32-a989-a07041e49557} <Debug> TCPHandler: Processed in 0.116539523 sec. | ||
| 330 | 2026.08.17 23:52:12.355579 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 331 | 2026.08.17 23:52:12.355661 [ 424014 ] {5068d503-88f2-4b81-a6ab-4061a167a7ff} <Trace> executeQuery: Query span trace_id for opentelemetry log: 33b4e22f-025a-a14b-ce4f-153b2a2ab065 | ||
| 332 | 2026.08.17 23:52:12.355817 [ 424014 ] {5068d503-88f2-4b81-a6ab-4061a167a7ff} <Debug> executeQuery: (from [::1]:56566) /* The field tables store named dimensions of each timeseries. | ||
| 333 | * | ||
| 334 | * As with the measurement tables, there is one field table for each field data | ||
| 335 | * type. Fields are deduplicated by using the "replacing merge tree", though | ||
| 336 | * this behavior **must not** be relied upon for query correctness. | ||
| 337 | * | ||
| 338 | * The index for the fields differs from the measurements, however. Rows are | ||
| 339 | * sorted by timeseries name, then field name, field value, and finally | ||
| 340 | * timeseries key. This reflects the most common pattern for looking them up: | ||
| 341 | * by field name and possibly value, within a timeseries. The resulting keys are | ||
| 342 | * usually then used to look up measurements. | ||
| 343 | * | ||
| 344 | * NOTE: We may want to consider a secondary index on these tables, sorting by | ||
| 345 | * timeseries name and then key, since it would improve lookups where one | ||
| 346 | * already has the key. Realistically though, these tables are quite small and | ||
| 347 | * so performance benefits will be low in absolute terms. | ||
| 348 | */ | ||
| 349 | CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local ON CLUSTER oximeter_cluster ( timeseries_name String, timeseries_key UInt64, field_name String, field_value Int64, last_updated_at DateTime MATERIALIZED now() ) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + INTERVAL 30 DAY (stage: Complete) | ||
| 350 | 2026.08.17 23:52:12.355863 [ 424014 ] {5068d503-88f2-4b81-a6ab-4061a167a7ff} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_i64_local | ||
| 351 | 2026.08.17 23:52:12.356029 [ 424014 ] {5068d503-88f2-4b81-a6ab-4061a167a7ff} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_i64_local | ||
| 352 | 2026.08.17 23:52:12.363749 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 353 | 2026.08.17 23:52:12.363762 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 354 | 2026.08.17 23:52:12.367955 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=5, queue_size=5, entries=query-0000000000..query-0000000004, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000003, last_skipped_entry_name=none | ||
| 355 | 2026.08.17 23:52:12.367960 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000004 | ||
| 356 | 2026.08.17 23:52:12.367963 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000004 | ||
| 357 | 2026.08.17 23:52:12.368201 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000004 (CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '2d79911c-8882-4309-a914-7e64c31972ae' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30)) | ||
| 358 | 2026.08.17 23:52:12.375972 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '2d79911c-8882-4309-a914-7e64c31972ae' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 359 | 2026.08.17 23:52:12.376008 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Trace> executeQuery: Query span trace_id for opentelemetry log: 33b4e22f-025a-a14b-ce4f-153b2a2ab065 | ||
| 360 | 2026.08.17 23:52:12.376085 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 5068d503-88f2-4b81-a6ab-4061a167a7ff) /* ddl_entry=query-0000000004 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '2d79911c-8882-4309-a914-7e64c31972ae' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) (stage: Complete) | ||
| 361 | 2026.08.17 23:52:12.382595 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Loading data parts | ||
| 362 | 2026.08.17 23:52:12.384029 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): There are no data parts | ||
| 363 | 2026.08.17 23:52:12.389322 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Creating table /clickhouse/tables/1/fields_i64_local | ||
| 364 | 2026.08.17 23:52:12.427571 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Creating shared ID for table oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) | ||
| 365 | 2026.08.17 23:52:12.427616 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Shared ID for table oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) doesn't exist in ZooKeeper on path /clickhouse/tables/1/fields_i64_local/table_shared_id | ||
| 366 | 2026.08.17 23:52:12.427620 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Got candidate ID 2d79911c-8882-4309-a914-7e64c31972ae, will try to create it in ZooKeeper on path /clickhouse/tables/1/fields_i64_local/table_shared_id | ||
| 367 | 2026.08.17 23:52:12.432403 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Initializing table shared ID with 2d79911c-8882-4309-a914-7e64c31972ae | ||
| 368 | 2026.08.17 23:52:12.437807 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Starting up table | ||
| 369 | 2026.08.17 23:52:12.441355 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Information> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Became leader | ||
| 370 | 2026.08.17 23:52:12.441360 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 371 | 2026.08.17 23:52:12.441391 [ 424604 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 372 | 2026.08.17 23:52:12.441406 [ 424604 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 373 | 2026.08.17 23:52:12.445434 [ 424604 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 374 | 2026.08.17 23:52:12.445467 [ 424604 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 375 | 2026.08.17 23:52:12.445472 [ 424604 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/fields_i64_local/replicas/1/queue | ||
| 376 | 2026.08.17 23:52:12.445503 [ 424604 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 377 | 2026.08.17 23:52:12.445536 [ 424604 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 378 | 2026.08.17 23:52:12.450917 [ 424604 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 379 | 2026.08.17 23:52:12.451002 [ 424602 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 6471ms | ||
| 380 | 2026.08.17 23:52:12.451090 [ 425097 ] {690e2c4e-cd82-45cf-9ad4-a7bee48e84d8} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000004 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '2d79911c-8882-4309-a914-7e64c31972ae' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 381 | 2026.08.17 23:52:12.459174 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 382 | 2026.08.17 23:52:12.518950 [ 424014 ] {5068d503-88f2-4b81-a6ab-4061a167a7ff} <Debug> executeQuery: Read 2 rows, 94.00 B in 0.163271 sec., 12.249572796148735 rows/sec., 575.73 B/sec. | ||
| 383 | 2026.08.17 23:52:12.520122 [ 424014 ] {5068d503-88f2-4b81-a6ab-4061a167a7ff} <Debug> TCPHandler: Processed in 0.164699548 sec. | ||
| 384 | 2026.08.17 23:52:12.520467 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 385 | 2026.08.17 23:52:12.520537 [ 424014 ] {092bb05c-359b-4ef3-8538-bc83d2646b53} <Trace> executeQuery: Query span trace_id for opentelemetry log: 282fbfc0-8e7d-73c6-e01c-fa5e0ad2a972 | ||
| 386 | 2026.08.17 23:52:12.520631 [ 424014 ] {092bb05c-359b-4ef3-8538-bc83d2646b53} <Debug> executeQuery: (from [::1]:56566) CREATE TABLE IF NOT EXISTS oximeter.fields_i64 ON CLUSTER oximeter_cluster AS oximeter.fields_i64_local ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 387 | 2026.08.17 23:52:12.520670 [ 424014 ] {092bb05c-359b-4ef3-8538-bc83d2646b53} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_i64 | ||
| 388 | 2026.08.17 23:52:12.520674 [ 424014 ] {092bb05c-359b-4ef3-8538-bc83d2646b53} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 389 | 2026.08.17 23:52:12.520679 [ 424014 ] {092bb05c-359b-4ef3-8538-bc83d2646b53} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 390 | 2026.08.17 23:52:12.520745 [ 424014 ] {092bb05c-359b-4ef3-8538-bc83d2646b53} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_i64 | ||
| 391 | 2026.08.17 23:52:12.520747 [ 424014 ] {092bb05c-359b-4ef3-8538-bc83d2646b53} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 392 | 2026.08.17 23:52:12.529126 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 393 | 2026.08.17 23:52:12.529138 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 394 | 2026.08.17 23:52:12.529244 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=6, queue_size=6, entries=query-0000000000..query-0000000005, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000004, last_skipped_entry_name=none | ||
| 395 | 2026.08.17 23:52:12.529248 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000005 | ||
| 396 | 2026.08.17 23:52:12.529260 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000005 | ||
| 397 | 2026.08.17 23:52:12.533275 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000005 (CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID '043624db-6f37-4b68-9c98-02ab5ccfc042' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 398 | 2026.08.17 23:52:12.551221 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID '043624db-6f37-4b68-9c98-02ab5ccfc042' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 399 | 2026.08.17 23:52:12.551264 [ 425097 ] {e93c9214-01b7-4763-b813-050a96002974} <Trace> executeQuery: Query span trace_id for opentelemetry log: 282fbfc0-8e7d-73c6-e01c-fa5e0ad2a972 | ||
| 400 | 2026.08.17 23:52:12.551333 [ 425097 ] {e93c9214-01b7-4763-b813-050a96002974} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 092bb05c-359b-4ef3-8538-bc83d2646b53) /* ddl_entry=query-0000000005 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID '043624db-6f37-4b68-9c98-02ab5ccfc042' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 401 | 2026.08.17 23:52:12.551677 [ 425097 ] {e93c9214-01b7-4763-b813-050a96002974} <Debug> StorageDistributed (fields_i64): Auto-increment is 0 | ||
| 402 | 2026.08.17 23:52:12.557682 [ 425097 ] {e93c9214-01b7-4763-b813-050a96002974} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000005 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID '043624db-6f37-4b68-9c98-02ab5ccfc042' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 403 | 2026.08.17 23:52:12.560823 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 404 | 2026.08.17 23:52:12.652496 [ 424014 ] {092bb05c-359b-4ef3-8538-bc83d2646b53} <Debug> executeQuery: Read 2 rows, 94.00 B in 0.13194 sec., 15.158405335758678 rows/sec., 712.45 B/sec. | ||
| 405 | 2026.08.17 23:52:12.652533 [ 424014 ] {092bb05c-359b-4ef3-8538-bc83d2646b53} <Debug> TCPHandler: Processed in 0.132243615 sec. | ||
| 406 | 2026.08.17 23:52:12.652950 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 407 | 2026.08.17 23:52:12.653023 [ 424014 ] {afbd441a-763b-4867-8336-bb878b20433c} <Trace> executeQuery: Query span trace_id for opentelemetry log: 379250d6-bc2d-effb-5fec-0dcccec218d1 | ||
| 408 | 2026.08.17 23:52:12.653156 [ 424014 ] {afbd441a-763b-4867-8336-bb878b20433c} <Debug> executeQuery: (from [::1]:56566) CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local ON CLUSTER oximeter_cluster ( timeseries_name String, timeseries_key UInt64, field_name String, field_value UUID, last_updated_at DateTime MATERIALIZED now() ) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + INTERVAL 30 DAY (stage: Complete) | ||
| 409 | 2026.08.17 23:52:12.653199 [ 424014 ] {afbd441a-763b-4867-8336-bb878b20433c} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_uuid_local | ||
| 410 | 2026.08.17 23:52:12.653378 [ 424014 ] {afbd441a-763b-4867-8336-bb878b20433c} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_uuid_local | ||
| 411 | 2026.08.17 23:52:12.662509 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 412 | 2026.08.17 23:52:12.662513 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 413 | 2026.08.17 23:52:12.666646 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=7, queue_size=7, entries=query-0000000000..query-0000000006, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000005, last_skipped_entry_name=none | ||
| 414 | 2026.08.17 23:52:12.666651 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000006 | ||
| 415 | 2026.08.17 23:52:12.666654 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000006 | ||
| 416 | 2026.08.17 23:52:12.666887 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000006 (CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID 'eaa5beef-895d-41af-915b-bd6fb56613c4' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30)) | ||
| 417 | 2026.08.17 23:52:12.675213 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID 'eaa5beef-895d-41af-915b-bd6fb56613c4' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 418 | 2026.08.17 23:52:12.675247 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Trace> executeQuery: Query span trace_id for opentelemetry log: 379250d6-bc2d-effb-5fec-0dcccec218d1 | ||
| 419 | 2026.08.17 23:52:12.675326 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: afbd441a-763b-4867-8336-bb878b20433c) /* ddl_entry=query-0000000006 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID 'eaa5beef-895d-41af-915b-bd6fb56613c4' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) (stage: Complete) | ||
| 420 | 2026.08.17 23:52:12.679956 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Loading data parts | ||
| 421 | 2026.08.17 23:52:12.680063 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): There are no data parts | ||
| 422 | 2026.08.17 23:52:12.685318 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Creating table /clickhouse/tables/1/fields_uuid_local | ||
| 423 | 2026.08.17 23:52:12.723168 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Creating shared ID for table oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) | ||
| 424 | 2026.08.17 23:52:12.723204 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Shared ID for table oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) doesn't exist in ZooKeeper on path /clickhouse/tables/1/fields_uuid_local/table_shared_id | ||
| 425 | 2026.08.17 23:52:12.723208 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Got candidate ID eaa5beef-895d-41af-915b-bd6fb56613c4, will try to create it in ZooKeeper on path /clickhouse/tables/1/fields_uuid_local/table_shared_id | ||
| 426 | 2026.08.17 23:52:12.730831 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Initializing table shared ID with eaa5beef-895d-41af-915b-bd6fb56613c4 | ||
| 427 | 2026.08.17 23:52:12.736004 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Starting up table | ||
| 428 | 2026.08.17 23:52:12.744194 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Information> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Became leader | ||
| 429 | 2026.08.17 23:52:12.744200 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 430 | 2026.08.17 23:52:12.744231 [ 424733 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 431 | 2026.08.17 23:52:12.744246 [ 424733 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 432 | 2026.08.17 23:52:12.751878 [ 424733 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 433 | 2026.08.17 23:52:12.751906 [ 424733 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 434 | 2026.08.17 23:52:12.751909 [ 424733 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/fields_uuid_local/replicas/1/queue | ||
| 435 | 2026.08.17 23:52:12.751937 [ 424733 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 436 | 2026.08.17 23:52:12.751968 [ 424733 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 437 | 2026.08.17 23:52:12.756356 [ 424733 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 438 | 2026.08.17 23:52:12.756440 [ 424764 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 6446ms | ||
| 439 | 2026.08.17 23:52:12.756507 [ 425097 ] {3cfe317a-2816-4127-a273-2f105a98e786} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000006 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID 'eaa5beef-895d-41af-915b-bd6fb56613c4' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 440 | 2026.08.17 23:52:12.760583 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 441 | 2026.08.17 23:52:12.817671 [ 424014 ] {afbd441a-763b-4867-8336-bb878b20433c} <Debug> executeQuery: Read 2 rows, 94.00 B in 0.164629 sec., 12.148527902131459 rows/sec., 570.98 B/sec. | ||
| 442 | 2026.08.17 23:52:12.817702 [ 424014 ] {afbd441a-763b-4867-8336-bb878b20433c} <Debug> TCPHandler: Processed in 0.164921828 sec. | ||
| 443 | 2026.08.17 23:52:12.818079 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 444 | 2026.08.17 23:52:12.819286 [ 424014 ] {23661bc2-14a7-494a-94cb-d7e9d83f0678} <Trace> executeQuery: Query span trace_id for opentelemetry log: ea78707a-6c67-f803-b167-bd71d8be8186 | ||
| 445 | 2026.08.17 23:52:12.819354 [ 424014 ] {23661bc2-14a7-494a-94cb-d7e9d83f0678} <Debug> executeQuery: (from [::1]:56566) CREATE TABLE IF NOT EXISTS oximeter.fields_uuid ON CLUSTER oximeter_cluster AS oximeter.fields_uuid_local ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 446 | 2026.08.17 23:52:12.819387 [ 424014 ] {23661bc2-14a7-494a-94cb-d7e9d83f0678} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_uuid | ||
| 447 | 2026.08.17 23:52:12.819391 [ 424014 ] {23661bc2-14a7-494a-94cb-d7e9d83f0678} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 448 | 2026.08.17 23:52:12.819396 [ 424014 ] {23661bc2-14a7-494a-94cb-d7e9d83f0678} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 449 | 2026.08.17 23:52:12.819448 [ 424014 ] {23661bc2-14a7-494a-94cb-d7e9d83f0678} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_uuid | ||
| 450 | 2026.08.17 23:52:12.819450 [ 424014 ] {23661bc2-14a7-494a-94cb-d7e9d83f0678} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 451 | 2026.08.17 23:52:12.826432 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 452 | 2026.08.17 23:52:12.826436 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 453 | 2026.08.17 23:52:12.826543 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=8, queue_size=8, entries=query-0000000000..query-0000000007, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000006, last_skipped_entry_name=none | ||
| 454 | 2026.08.17 23:52:12.826551 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000007 | ||
| 455 | 2026.08.17 23:52:12.826554 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000007 | ||
| 456 | 2026.08.17 23:52:12.831368 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000007 (CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '9aa0f265-14c6-42dd-8a43-e4bc546a6053' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 457 | 2026.08.17 23:52:12.838148 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '9aa0f265-14c6-42dd-8a43-e4bc546a6053' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 458 | 2026.08.17 23:52:12.838188 [ 425097 ] {1186e1b5-57e0-4f19-b36b-c6e997c73e17} <Trace> executeQuery: Query span trace_id for opentelemetry log: ea78707a-6c67-f803-b167-bd71d8be8186 | ||
| 459 | 2026.08.17 23:52:12.838258 [ 425097 ] {1186e1b5-57e0-4f19-b36b-c6e997c73e17} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 23661bc2-14a7-494a-94cb-d7e9d83f0678) /* ddl_entry=query-0000000007 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '9aa0f265-14c6-42dd-8a43-e4bc546a6053' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 460 | 2026.08.17 23:52:12.838557 [ 425097 ] {1186e1b5-57e0-4f19-b36b-c6e997c73e17} <Debug> StorageDistributed (fields_uuid): Auto-increment is 0 | ||
| 461 | 2026.08.17 23:52:12.843681 [ 425097 ] {1186e1b5-57e0-4f19-b36b-c6e997c73e17} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000007 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '9aa0f265-14c6-42dd-8a43-e4bc546a6053' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 462 | 2026.08.17 23:52:12.851860 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 463 | 2026.08.17 23:52:12.931946 [ 424014 ] {23661bc2-14a7-494a-94cb-d7e9d83f0678} <Debug> executeQuery: Read 2 rows, 94.00 B in 0.112641 sec., 17.755524187462825 rows/sec., 834.51 B/sec. | ||
| 464 | 2026.08.17 23:52:12.931976 [ 424014 ] {23661bc2-14a7-494a-94cb-d7e9d83f0678} <Debug> TCPHandler: Processed in 0.114014442 sec. | ||
| 465 | 2026.08.17 23:52:12.932187 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 466 | 2026.08.17 23:52:12.932231 [ 424014 ] {8842ae81-e109-4c7a-8418-94395b79df83} <Trace> executeQuery: Query span trace_id for opentelemetry log: e81593ca-7d25-7828-5db4-551f33b77ef3 | ||
| 467 | 2026.08.17 23:52:12.932368 [ 424014 ] {8842ae81-e109-4c7a-8418-94395b79df83} <Debug> executeQuery: (from [::1]:56566) /* The timeseries schema table stores the extracted schema for the samples | ||
| 468 | * oximeter collects. | ||
| 469 | */ | ||
| 470 | CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local ON CLUSTER oximeter_cluster ( timeseries_name String, fields Nested( name String, type Enum( 'Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13 ), source Enum( 'Target' = 1, 'Metric' = 2 ) ), datum_type Enum( 'Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27 ), created DateTime64(9, 'UTC') ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) (stage: Complete) | ||
| 471 | 2026.08.17 23:52:12.932414 [ 424014 ] {8842ae81-e109-4c7a-8418-94395b79df83} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.timeseries_schema_local | ||
| 472 | 2026.08.17 23:52:12.932587 [ 424014 ] {8842ae81-e109-4c7a-8418-94395b79df83} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.timeseries_schema_local | ||
| 473 | 2026.08.17 23:52:12.939445 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 474 | 2026.08.17 23:52:12.939460 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 475 | 2026.08.17 23:52:12.942125 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=9, queue_size=9, entries=query-0000000000..query-0000000008, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000007, last_skipped_entry_name=none | ||
| 476 | 2026.08.17 23:52:12.942133 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000008 | ||
| 477 | 2026.08.17 23:52:12.942137 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000008 | ||
| 478 | 2026.08.17 23:52:12.942561 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000008 (CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID '05a0062d-b532-435a-91d7-b9da99374f1d' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name)) | ||
| 479 | 2026.08.17 23:52:12.949727 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID '05a0062d-b532-435a-91d7-b9da99374f1d' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) | ||
| 480 | 2026.08.17 23:52:12.949779 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Trace> executeQuery: Query span trace_id for opentelemetry log: e81593ca-7d25-7828-5db4-551f33b77ef3 | ||
| 481 | 2026.08.17 23:52:12.949893 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 8842ae81-e109-4c7a-8418-94395b79df83) /* ddl_entry=query-0000000008 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID '05a0062d-b532-435a-91d7-b9da99374f1d' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) (stage: Complete) | ||
| 482 | 2026.08.17 23:52:12.953847 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Loading data parts | ||
| 483 | 2026.08.17 23:52:12.953970 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): There are no data parts | ||
| 484 | 2026.08.17 23:52:12.958476 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Creating table /clickhouse/tables/1/timeseries_schema_local | ||
| 485 | 2026.08.17 23:52:12.965947 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Information> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): It looks like the table /clickhouse/tables/1/timeseries_schema_local was created by another server at the same moment, will retry | ||
| 486 | 2026.08.17 23:52:12.966034 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): This table /clickhouse/tables/1/timeseries_schema_local is already created, will add new replica | ||
| 487 | 2026.08.17 23:52:12.966482 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Creating replica /clickhouse/tables/1/timeseries_schema_local/replicas/1 | ||
| 488 | 2026.08.17 23:52:13.000850 [ 425094 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 378.11 MiB, peak 379.36 MiB, free memory in arenas 7.76 MiB, will set to 379.42 MiB (RSS), difference: 1.31 MiB | ||
| 489 | 2026.08.17 23:52:13.013201 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Creating shared ID for table oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) | ||
| 490 | 2026.08.17 23:52:13.013324 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Initializing table shared ID with 05a0062d-b532-435a-91d7-b9da99374f1d | ||
| 491 | 2026.08.17 23:52:13.019476 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Starting up table | ||
| 492 | 2026.08.17 23:52:13.019611 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Information> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Became leader | ||
| 493 | 2026.08.17 23:52:13.019614 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 494 | 2026.08.17 23:52:13.019648 [ 424739 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 495 | 2026.08.17 23:52:13.019668 [ 424739 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 496 | 2026.08.17 23:52:13.026547 [ 424739 ] {} <Information> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Replica 2 has log pointer '', approximate 1 queue lag and 0 queue size | ||
| 497 | 2026.08.17 23:52:13.026566 [ 424739 ] {} <Information> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Will mimic 2 | ||
| 498 | 2026.08.17 23:52:13.034091 [ 424739 ] {} <Warning> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Log pointer of source replica 2 changed while we loading queue nodes. Will retry. | ||
| 499 | 2026.08.17 23:52:13.038619 [ 424739 ] {} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Queued 0 parts to be fetched, 0 parts ignored | ||
| 500 | 2026.08.17 23:52:13.038626 [ 424739 ] {} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Copied 0 queue entries, 0 entries ignored | ||
| 501 | 2026.08.17 23:52:13.038656 [ 424739 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Parts in ZooKeeper after mimic: | ||
| 502 | 2026.08.17 23:52:13.038658 [ 424739 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Enqueued fetches after mimic: | ||
| 503 | 2026.08.17 23:52:13.046974 [ 424739 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 504 | 2026.08.17 23:52:13.047019 [ 424739 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 505 | 2026.08.17 23:52:13.047022 [ 424739 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/timeseries_schema_local/replicas/1/queue | ||
| 506 | 2026.08.17 23:52:13.047066 [ 424739 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 507 | 2026.08.17 23:52:13.047111 [ 424739 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 508 | 2026.08.17 23:52:13.047423 [ 424739 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 509 | 2026.08.17 23:52:13.047467 [ 424750 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 6532ms | ||
| 510 | 2026.08.17 23:52:13.047572 [ 425097 ] {8ade62b9-8e01-46cb-a675-6f71e16b465c} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000008 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID '05a0062d-b532-435a-91d7-b9da99374f1d' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) | ||
| 511 | 2026.08.17 23:52:13.051388 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 512 | 2026.08.17 23:52:13.097659 [ 424014 ] {8842ae81-e109-4c7a-8418-94395b79df83} <Debug> executeQuery: Read 2 rows, 94.00 B in 0.16541 sec., 12.091167402212683 rows/sec., 568.28 B/sec. | ||
| 513 | 2026.08.17 23:52:13.097695 [ 424014 ] {8842ae81-e109-4c7a-8418-94395b79df83} <Debug> TCPHandler: Processed in 0.165566468 sec. | ||
| 514 | 2026.08.17 23:52:13.098003 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 515 | 2026.08.17 23:52:13.098066 [ 424014 ] {ba34d095-e353-42cb-bfc7-790d09f3d772} <Trace> executeQuery: Query span trace_id for opentelemetry log: 4f2d7010-71a7-d310-28c8-535cafa3c1c9 | ||
| 516 | 2026.08.17 23:52:13.098158 [ 424014 ] {ba34d095-e353-42cb-bfc7-790d09f3d772} <Debug> executeQuery: (from [::1]:56566) CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema ON CLUSTER oximeter_cluster AS oximeter.timeseries_schema_local ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 517 | 2026.08.17 23:52:13.098197 [ 424014 ] {ba34d095-e353-42cb-bfc7-790d09f3d772} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.timeseries_schema | ||
| 518 | 2026.08.17 23:52:13.098208 [ 424014 ] {ba34d095-e353-42cb-bfc7-790d09f3d772} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 519 | 2026.08.17 23:52:13.098214 [ 424014 ] {ba34d095-e353-42cb-bfc7-790d09f3d772} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 520 | 2026.08.17 23:52:13.098363 [ 424014 ] {ba34d095-e353-42cb-bfc7-790d09f3d772} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.timeseries_schema | ||
| 521 | 2026.08.17 23:52:13.099333 [ 424014 ] {ba34d095-e353-42cb-bfc7-790d09f3d772} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 522 | 2026.08.17 23:52:13.108094 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 523 | 2026.08.17 23:52:13.108099 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 524 | 2026.08.17 23:52:13.112671 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=10, queue_size=10, entries=query-0000000000..query-0000000009, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000008, last_skipped_entry_name=none | ||
| 525 | 2026.08.17 23:52:13.112683 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000009 | ||
| 526 | 2026.08.17 23:52:13.112686 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000009 | ||
| 527 | 2026.08.17 23:52:13.113076 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000009 (CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID 'e7ae042c-f627-4acc-8fc7-b9453761784f' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 528 | 2026.08.17 23:52:13.119590 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID 'e7ae042c-f627-4acc-8fc7-b9453761784f' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 529 | 2026.08.17 23:52:13.119634 [ 425097 ] {de47742b-2055-48f7-8f71-80eb2e6fa3be} <Trace> executeQuery: Query span trace_id for opentelemetry log: 4f2d7010-71a7-d310-28c8-535cafa3c1c9 | ||
| 530 | 2026.08.17 23:52:13.119741 [ 425097 ] {de47742b-2055-48f7-8f71-80eb2e6fa3be} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: ba34d095-e353-42cb-bfc7-790d09f3d772) /* ddl_entry=query-0000000009 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID 'e7ae042c-f627-4acc-8fc7-b9453761784f' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 531 | 2026.08.17 23:52:13.120130 [ 425097 ] {de47742b-2055-48f7-8f71-80eb2e6fa3be} <Debug> StorageDistributed (timeseries_schema): Auto-increment is 0 | ||
| 532 | 2026.08.17 23:52:13.126498 [ 425097 ] {de47742b-2055-48f7-8f71-80eb2e6fa3be} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000009 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID 'e7ae042c-f627-4acc-8fc7-b9453761784f' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 533 | 2026.08.17 23:52:13.133115 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 534 | 2026.08.17 23:52:13.213438 [ 424014 ] {ba34d095-e353-42cb-bfc7-790d09f3d772} <Debug> executeQuery: Read 2 rows, 94.00 B in 0.115348 sec., 17.33883552380622 rows/sec., 814.93 B/sec. | ||
| 535 | 2026.08.17 23:52:13.213553 [ 424014 ] {ba34d095-e353-42cb-bfc7-790d09f3d772} <Debug> TCPHandler: Processed in 0.115700573 sec. | ||
| 536 | 2026.08.17 23:52:13.214039 [ 424016 ] {} <Trace> TCP-Session: 31d2804f-7a6a-4904-a91f-9b4e4cd6922d Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 537 | 2026.08.17 23:52:13.214157 [ 424016 ] {86149ad3-4d40-4257-be6d-29733fc9d309} <Trace> executeQuery: Query span trace_id for opentelemetry log: 0f371af8-b039-1aa9-6161-ba671deb94c0 | ||
| 538 | 2026.08.17 23:52:13.214240 [ 424016 ] {86149ad3-4d40-4257-be6d-29733fc9d309} <Debug> executeQuery: (from [::1]:56572) SELECT name FROM system.tables WHERE database = 'oximeter' AND engine = 'ReplicatedMergeTree' (stage: Complete) | ||
| 539 | 2026.08.17 23:52:13.214561 [ 424016 ] {86149ad3-4d40-4257-be6d-29733fc9d309} <Trace> ContextAccess (default): Access granted: SELECT(database, name, engine) ON system.tables | ||
| 540 | 2026.08.17 23:52:13.214677 [ 424016 ] {86149ad3-4d40-4257-be6d-29733fc9d309} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 541 | 2026.08.17 23:52:13.214912 [ 429503 ] {86149ad3-4d40-4257-be6d-29733fc9d309} <Trace> ContextAccess (default): Access granted: SHOW TABLES ON *.* | ||
| 542 | 2026.08.17 23:52:13.216176 [ 424016 ] {86149ad3-4d40-4257-be6d-29733fc9d309} <Debug> executeQuery: Read 9 rows, 631.00 B in 0.002003 sec., 4493.260109835247 rows/sec., 307.64 KiB/sec. | ||
| 543 | 2026.08.17 23:52:13.216201 [ 424016 ] {86149ad3-4d40-4257-be6d-29733fc9d309} <Debug> TCPHandler: Processed in 0.002333632 sec. | ||
| 544 | 2026.08.17 23:52:13.240991 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 545 | 2026.08.17 23:52:13.241069 [ 424014 ] {5bd6ad7e-e499-4524-b630-45c5e091a6f4} <Trace> executeQuery: Query span trace_id for opentelemetry log: 3ed26373-b136-8acb-3d5e-7377460e6dbe | ||
| 546 | 2026.08.17 23:52:13.241136 [ 424014 ] {5bd6ad7e-e499-4524-b630-45c5e091a6f4} <Debug> executeQuery: (from [::1]:56566) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 547 | 2026.08.17 23:52:13.241431 [ 424014 ] {5bd6ad7e-e499-4524-b630-45c5e091a6f4} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 548 | 2026.08.17 23:52:13.241633 [ 424014 ] {5bd6ad7e-e499-4524-b630-45c5e091a6f4} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 549 | 2026.08.17 23:52:13.241796 [ 424014 ] {5bd6ad7e-e499-4524-b630-45c5e091a6f4} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 550 | 2026.08.17 23:52:13.241818 [ 424014 ] {5bd6ad7e-e499-4524-b630-45c5e091a6f4} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 551 | 2026.08.17 23:52:13.241860 [ 424014 ] {5bd6ad7e-e499-4524-b630-45c5e091a6f4} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 552 | 2026.08.17 23:52:13.242175 [ 424014 ] {5bd6ad7e-e499-4524-b630-45c5e091a6f4} <Debug> TCPHandler: Processed in 0.001254491 sec. | ||
| 553 | 2026.08.17 23:52:13.249998 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 554 | 2026.08.17 23:52:13.250054 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Trace> executeQuery: Query span trace_id for opentelemetry log: 39971602-5437-b2f2-608a-8397e4ee59f6 | ||
| 555 | 2026.08.17 23:52:13.250084 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Debug> executeQuery: (from [::1]:56566) INSERT INTO oximeter.timeseries_schema FORMAT Native (stage: Complete) | ||
| 556 | 2026.08.17 23:52:13.250110 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 557 | 2026.08.17 23:52:13.250789 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 558 | 2026.08.17 23:52:13.250936 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 559 | 2026.08.17 23:52:13.250950 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.96 GiB. | ||
| 560 | 2026.08.17 23:52:13.251235 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Trace> MergedBlockOutputStream: filled checksums all_1_1_0 (state Temporary) | ||
| 561 | 2026.08.17 23:52:13.251331 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (Replicated OutputStream): Wrote block with ID 'all_5928087323661188991_6187416097055753511', 1 rows | ||
| 562 | 2026.08.17 23:52:13.257775 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Renaming temporary part tmp_insert_all_1_1_0 to all_0_0_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 563 | 2026.08.17 23:52:13.262638 [ 424640 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 8493ms | ||
| 564 | 2026.08.17 23:52:13.262810 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.012739 sec., 78.4990972603815 rows/sec., 9.66 KiB/sec. | ||
| 565 | 2026.08.17 23:52:13.262867 [ 424711 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000000 - log-0000000000 | ||
| 566 | 2026.08.17 23:52:13.263870 [ 424014 ] {89e44299-d1c4-40c7-ba7d-89d07438e8ff} <Debug> TCPHandler: Processed in 0.01392384 sec. | ||
| 567 | 2026.08.17 23:52:13.264068 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 568 | 2026.08.17 23:52:13.264124 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Trace> executeQuery: Query span trace_id for opentelemetry log: 3e763bd2-e99a-fa7d-0c3a-7c18da8ca4bb | ||
| 569 | 2026.08.17 23:52:13.264163 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Debug> executeQuery: (from [::1]:56566) INSERT INTO oximeter.fields_i64 FORMAT Native (stage: Complete) | ||
| 570 | 2026.08.17 23:52:13.264191 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 571 | 2026.08.17 23:52:13.264652 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64_local | ||
| 572 | 2026.08.17 23:52:13.264854 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 573 | 2026.08.17 23:52:13.266125 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.96 GiB. | ||
| 574 | 2026.08.17 23:52:13.266319 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Trace> MergedBlockOutputStream: filled checksums all_1_1_0 (state Temporary) | ||
| 575 | 2026.08.17 23:52:13.266427 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Replicated OutputStream): Wrote block with ID 'all_15662889916179982120_12555237044298603210', 8 rows | ||
| 576 | 2026.08.17 23:52:13.271571 [ 424711 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000000 to queue with type GET_PART with virtual parts [all_0_0_0] | ||
| 577 | 2026.08.17 23:52:13.271582 [ 424711 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 578 | 2026.08.17 23:52:13.272041 [ 424015 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47250, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 579 | 2026.08.17 23:52:13.272860 [ 424015 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ftimeseries_schema_local%2Freplicas%2F1&part=all_0_0_0&client_protocol_version=8&compress=false | ||
| 580 | 2026.08.17 23:52:13.272871 [ 424015 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (Replicated PartsService): Sending part all_0_0_0 | ||
| 581 | 2026.08.17 23:52:13.273098 [ 424015 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 582 | 2026.08.17 23:52:13.282942 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Renaming temporary part tmp_insert_all_1_1_0 to all_0_0_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 583 | 2026.08.17 23:52:13.291718 [ 424718 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 7915ms | ||
| 584 | 2026.08.17 23:52:13.291883 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Debug> executeQuery: Read 8 rows, 512.00 B in 0.027744 sec., 288.3506343713956 rows/sec., 18.02 KiB/sec. | ||
| 585 | 2026.08.17 23:52:13.291921 [ 424014 ] {93e89b70-fae9-4b31-aba7-8ca83d1304ce} <Debug> TCPHandler: Processed in 0.02792739 sec. | ||
| 586 | 2026.08.17 23:52:13.292213 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 587 | 2026.08.17 23:52:13.292280 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Trace> executeQuery: Query span trace_id for opentelemetry log: 29ebcb67-0223-ada4-3318-f6fa49e81acb | ||
| 588 | 2026.08.17 23:52:13.292317 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Debug> executeQuery: (from [::1]:56566) INSERT INTO oximeter.fields_uuid FORMAT Native (stage: Complete) | ||
| 589 | 2026.08.17 23:52:13.292355 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 590 | 2026.08.17 23:52:13.292788 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid_local | ||
| 591 | 2026.08.17 23:52:13.292996 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 592 | 2026.08.17 23:52:13.293009 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.96 GiB. | ||
| 593 | 2026.08.17 23:52:13.293212 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Trace> MergedBlockOutputStream: filled checksums all_1_1_0 (state Temporary) | ||
| 594 | 2026.08.17 23:52:13.293312 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Replicated OutputStream): Wrote block with ID 'all_6989642273983557071_11614382679588548598', 16 rows | ||
| 595 | 2026.08.17 23:52:13.304574 [ 424689 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000000 - log-0000000000 | ||
| 596 | 2026.08.17 23:52:13.312852 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Renaming temporary part tmp_insert_all_1_1_0 to all_0_0_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 597 | 2026.08.17 23:52:13.313022 [ 424015 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47250, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 598 | 2026.08.17 23:52:13.313053 [ 424015 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_i64_local%2Freplicas%2F1&part=all_0_0_0&client_protocol_version=8&compress=false | ||
| 599 | 2026.08.17 23:52:13.313061 [ 424015 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Replicated PartsService): Sending part all_0_0_0 | ||
| 600 | 2026.08.17 23:52:13.313242 [ 424015 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 601 | 2026.08.17 23:52:13.323011 [ 424689 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000000 to queue with type GET_PART with virtual parts [all_0_0_0] | ||
| 602 | 2026.08.17 23:52:13.323022 [ 424689 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 603 | 2026.08.17 23:52:13.329949 [ 424770 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 7985ms | ||
| 604 | 2026.08.17 23:52:13.330127 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Debug> executeQuery: Read 16 rows, 1.20 KiB in 0.03783 sec., 422.94475284166003 rows/sec., 31.60 KiB/sec. | ||
| 605 | 2026.08.17 23:52:13.330169 [ 424014 ] {6a831dba-f17f-4fe6-ab0c-eebba450b951} <Debug> TCPHandler: Processed in 0.038055788 sec. | ||
| 606 | 2026.08.17 23:52:13.330620 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 607 | 2026.08.17 23:52:13.330686 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Trace> executeQuery: Query span trace_id for opentelemetry log: 5b0921b5-bdd2-4738-36e5-de3e80310d9d | ||
| 608 | 2026.08.17 23:52:13.330740 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Debug> executeQuery: (from [::1]:56566) INSERT INTO oximeter.measurements_cumulativef64 FORMAT Native (stage: Complete) | ||
| 609 | 2026.08.17 23:52:13.330787 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 610 | 2026.08.17 23:52:13.331343 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64_local | ||
| 611 | 2026.08.17 23:52:13.331511 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 612 | 2026.08.17 23:52:13.331524 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.96 GiB. | ||
| 613 | 2026.08.17 23:52:13.333339 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Trace> MergedBlockOutputStream: filled checksums all_1_1_0 (state Temporary) | ||
| 614 | 2026.08.17 23:52:13.333445 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Replicated OutputStream): Wrote block with ID 'all_14117851236667471686_14030098646793254649', 400 rows | ||
| 615 | 2026.08.17 23:52:13.351381 [ 424015 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47250, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 616 | 2026.08.17 23:52:13.351412 [ 424015 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_uuid_local%2Freplicas%2F1&part=all_0_0_0&client_protocol_version=8&compress=false | ||
| 617 | 2026.08.17 23:52:13.351420 [ 424015 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Replicated PartsService): Sending part all_0_0_0 | ||
| 618 | 2026.08.17 23:52:13.351602 [ 424015 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 619 | 2026.08.17 23:52:13.367223 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Renaming temporary part tmp_insert_all_1_1_0 to all_0_0_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 620 | 2026.08.17 23:52:13.367222 [ 424747 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000000 - log-0000000000 | ||
| 621 | 2026.08.17 23:52:13.375197 [ 424743 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 7866ms | ||
| 622 | 2026.08.17 23:52:13.375328 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Debug> executeQuery: Read 400 rows, 25.78 KiB in 0.044628 sec., 8962.982880702697 rows/sec., 577.69 KiB/sec. | ||
| 623 | 2026.08.17 23:52:13.375358 [ 424014 ] {3fc667b5-3d05-434d-bbf5-f613cc93f39a} <Debug> TCPHandler: Processed in 0.044882252 sec. | ||
| 624 | 2026.08.17 23:52:13.377637 [ 424016 ] {} <Trace> TCP-Session: 31d2804f-7a6a-4904-a91f-9b4e4cd6922d Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 625 | 2026.08.17 23:52:13.378656 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> executeQuery: Query span trace_id for opentelemetry log: 97b46dae-dbc9-23c2-97c9-77d5711b2dd1 | ||
| 626 | 2026.08.17 23:52:13.378715 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Debug> executeQuery: (from [::1]:56572) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 627 | 2026.08.17 23:52:13.378972 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 628 | 2026.08.17 23:52:13.379025 [ 424747 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000000 to queue with type GET_PART with virtual parts [all_0_0_0] | ||
| 629 | 2026.08.17 23:52:13.379035 [ 424747 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 630 | 2026.08.17 23:52:13.379111 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 631 | 2026.08.17 23:52:13.379193 [ 424772 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000000 - log-0000000000 | ||
| 632 | 2026.08.17 23:52:13.379237 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 633 | 2026.08.17 23:52:13.379244 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name IN ('virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 634 | 2026.08.17 23:52:13.379323 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 635 | 2026.08.17 23:52:13.379357 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 636 | 2026.08.17 23:52:13.379391 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 637 | 2026.08.17 23:52:13.379472 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Key condition: (column 0 in 1-element set) | ||
| 638 | 2026.08.17 23:52:13.379484 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 639 | 2026.08.17 23:52:13.379499 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 640 | 2026.08.17 23:52:13.379503 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 641 | 2026.08.17 23:52:13.379506 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found continuous range in 2 steps | ||
| 642 | 2026.08.17 23:52:13.379518 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 643 | 2026.08.17 23:52:13.379523 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 644 | 2026.08.17 23:52:13.379543 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "in(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 645 | 2026.08.17 23:52:13.379551 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 1 rows starting from 0 | ||
| 646 | 2026.08.17 23:52:13.380279 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.00161 sec., 621.1180124223602 rows/sec., 76.43 KiB/sec. | ||
| 647 | 2026.08.17 23:52:13.380342 [ 424016 ] {5f7531b6-155f-442a-8889-9ccaa87558d1} <Debug> TCPHandler: Processed in 0.002791842 sec. | ||
| 648 | 2026.08.17 23:52:13.380844 [ 424016 ] {} <Trace> TCP-Session: 31d2804f-7a6a-4904-a91f-9b4e4cd6922d Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 649 | 2026.08.17 23:52:13.380885 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> executeQuery: Query span trace_id for opentelemetry log: 7f5e2bfb-45da-4bc6-0a47-cc9589002aa8 | ||
| 650 | 2026.08.17 23:52:13.380929 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Debug> executeQuery: (from [::1]:56572) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 651 | 2026.08.17 23:52:13.381086 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 652 | 2026.08.17 23:52:13.381180 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 653 | 2026.08.17 23:52:13.381269 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 654 | 2026.08.17 23:52:13.381275 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name IN ('virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 655 | 2026.08.17 23:52:13.381330 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 656 | 2026.08.17 23:52:13.381348 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 657 | 2026.08.17 23:52:13.381373 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 658 | 2026.08.17 23:52:13.381434 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Key condition: (column 0 in 1-element set) | ||
| 659 | 2026.08.17 23:52:13.381440 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 660 | 2026.08.17 23:52:13.381447 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 661 | 2026.08.17 23:52:13.381451 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 662 | 2026.08.17 23:52:13.381454 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found continuous range in 2 steps | ||
| 663 | 2026.08.17 23:52:13.381460 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 664 | 2026.08.17 23:52:13.381463 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 665 | 2026.08.17 23:52:13.381480 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "in(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 666 | 2026.08.17 23:52:13.381483 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 1 rows starting from 0 | ||
| 667 | 2026.08.17 23:52:13.381987 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.00109 sec., 917.4311926605504 rows/sec., 112.89 KiB/sec. | ||
| 668 | 2026.08.17 23:52:13.382736 [ 424016 ] {79cf28fe-8312-4b8f-b4db-e5867320b0a5} <Debug> TCPHandler: Processed in 0.001938281 sec. | ||
| 669 | 2026.08.17 23:52:13.383116 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 670 | 2026.08.17 23:52:13.383173 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> executeQuery: Query span trace_id for opentelemetry log: a97fff72-8238-b9f1-991d-9ae9d7131f36 | ||
| 671 | 2026.08.17 23:52:13.383295 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> executeQuery: (from [::1]:56566) SELECT fields_i64_pivot.cpu_id AS cpu_id, fields_uuid_pivot.instance_id AS instance_id, fields_uuid_pivot.project_id AS project_id, fields_uuid_pivot.timeseries_key AS timeseries_key FROM ( SELECT timeseries_key, anyIf(field_value, field_name = 'instance_id') AS instance_id, anyIf(field_value, field_name = 'project_id') AS project_id FROM oximeter.fields_uuid WHERE timeseries_name = 'virtual_machine:cpu_busy' GROUP BY timeseries_key ) AS fields_uuid_pivot JOIN ( SELECT timeseries_key, anyIf(field_value, field_name = 'cpu_id') AS cpu_id FROM oximeter.fields_i64 WHERE timeseries_name = 'virtual_machine:cpu_busy' GROUP BY timeseries_key ) AS fields_i64_pivot ON fields_uuid_pivot.timeseries_key = fields_i64_pivot.timeseries_key (stage: Complete) | ||
| 672 | 2026.08.17 23:52:13.383593 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 673 | 2026.08.17 23:52:13.383727 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 674 | 2026.08.17 23:52:13.383847 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 675 | 2026.08.17 23:52:13.385549 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 676 | 2026.08.17 23:52:13.385659 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 677 | 2026.08.17 23:52:13.385766 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 678 | 2026.08.17 23:52:13.385772 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name = 'virtual_machine:cpu_busy'" moved to PREWHERE | ||
| 679 | 2026.08.17 23:52:13.385828 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64_local | ||
| 680 | 2026.08.17 23:52:13.385851 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 681 | 2026.08.17 23:52:13.385878 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 682 | 2026.08.17 23:52:13.385914 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> HashJoin: (0x7fd7b515b318) Datatype: EMPTY, kind: Inner, strictness: All, right header: fields_i64_pivot.timeseries_key UInt64 UInt64(size = 0), cpu_id Int64 Int64(size = 0) | ||
| 683 | 2026.08.17 23:52:13.385923 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> HashJoin: (0x7fd7b515b318) Keys: [(timeseries_key) = (fields_i64_pivot.timeseries_key)] | ||
| 684 | 2026.08.17 23:52:13.386101 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 685 | 2026.08.17 23:52:13.388042 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 686 | 2026.08.17 23:52:13.388164 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 687 | 2026.08.17 23:52:13.388170 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name = 'virtual_machine:cpu_busy'" moved to PREWHERE | ||
| 688 | 2026.08.17 23:52:13.388241 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid_local | ||
| 689 | 2026.08.17 23:52:13.388259 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 690 | 2026.08.17 23:52:13.388287 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 691 | 2026.08.17 23:52:13.388290 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 692 | 2026.08.17 23:52:13.388301 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> JoiningTransform: Before join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0)' | ||
| 693 | 2026.08.17 23:52:13.388315 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> JoiningTransform: After join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0), cpu_id Int64 Int64(size = 0)' | ||
| 694 | 2026.08.17 23:52:13.388619 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Key condition: (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']) | ||
| 695 | 2026.08.17 23:52:13.388631 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 696 | 2026.08.17 23:52:13.388645 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 697 | 2026.08.17 23:52:13.388648 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 698 | 2026.08.17 23:52:13.388652 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 699 | 2026.08.17 23:52:13.388660 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 700 | 2026.08.17 23:52:13.388663 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 701 | 2026.08.17 23:52:13.388681 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 702 | 2026.08.17 23:52:13.388688 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 16 rows starting from 0 | ||
| 703 | 2026.08.17 23:52:13.388785 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Key condition: (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']) | ||
| 704 | 2026.08.17 23:52:13.388791 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 705 | 2026.08.17 23:52:13.388795 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 706 | 2026.08.17 23:52:13.388798 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 707 | 2026.08.17 23:52:13.388801 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 708 | 2026.08.17 23:52:13.388804 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 709 | 2026.08.17 23:52:13.388807 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 710 | 2026.08.17 23:52:13.388814 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 711 | 2026.08.17 23:52:13.388817 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 8 rows starting from 0 | ||
| 712 | 2026.08.17 23:52:13.388897 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> JoiningTransform: Before join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0)' | ||
| 713 | 2026.08.17 23:52:13.388907 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> JoiningTransform: After join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0), cpu_id Int64 Int64(size = 0)' | ||
| 714 | 2026.08.17 23:52:13.389933 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> AggregatingTransform: Aggregating | ||
| 715 | 2026.08.17 23:52:13.389951 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> Aggregator: Aggregation method: key64 | ||
| 716 | 2026.08.17 23:52:13.389984 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.00112732 sec. (7096.477 rows/sec., 117.81 KiB/sec.) | ||
| 717 | 2026.08.17 23:52:13.389987 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> Aggregator: Merging aggregated data | ||
| 718 | 2026.08.17 23:52:13.389990 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> Aggregator: Statistics updated for key=1559201803930241434: new sum_of_sizes=8, median_size=8 | ||
| 719 | 2026.08.17 23:52:13.394716 [ 424015 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47250, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 720 | 2026.08.17 23:52:13.394749 [ 424015 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Fmeasurements_cumulativef64_local%2Freplicas%2F1&part=all_0_0_0&client_protocol_version=8&compress=false | ||
| 721 | 2026.08.17 23:52:13.394757 [ 424015 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Replicated PartsService): Sending part all_0_0_0 | ||
| 722 | 2026.08.17 23:52:13.398521 [ 424772 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000000 to queue with type GET_PART with virtual parts [all_0_0_0] | ||
| 723 | 2026.08.17 23:52:13.398536 [ 424772 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 724 | 2026.08.17 23:52:13.415352 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> AggregatingTransform: Aggregating | ||
| 725 | 2026.08.17 23:52:13.415940 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> Aggregator: Aggregation method: key64 | ||
| 726 | 2026.08.17 23:52:13.415972 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.02723879 sec. (587.398 rows/sec., 14.91 KiB/sec.) | ||
| 727 | 2026.08.17 23:52:13.415975 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> Aggregator: Merging aggregated data | ||
| 728 | 2026.08.17 23:52:13.415980 [ 426210 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> Aggregator: Statistics updated for key=11757313445726935263: new sum_of_sizes=8, median_size=8 | ||
| 729 | 2026.08.17 23:52:13.416052 [ 424015 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 730 | 2026.08.17 23:52:13.416993 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> executeQuery: Read 24 rows, 1.70 KiB in 0.03379 sec., 710.2693104468777 rows/sec., 50.17 KiB/sec. | ||
| 731 | 2026.08.17 23:52:13.417188 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Trace> HashJoin: (0x7fd7b515b318) Join data is being destroyed, 12544 bytes and 8 rows in hash table | ||
| 732 | 2026.08.17 23:52:13.417321 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> MemoryTracker: Peak memory usage (for query): 183.48 KiB. | ||
| 733 | 2026.08.17 23:52:13.417325 [ 424014 ] {b66815cf-3112-4a00-92d9-e3e66ee5cab3} <Debug> TCPHandler: Processed in 0.034256185 sec. | ||
| 734 | 2026.08.17 23:52:13.417949 [ 424014 ] {} <Trace> TCP-Session: 93c7fca1-386b-4727-b021-6dfe1d54516b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 735 | 2026.08.17 23:52:13.418022 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> executeQuery: Query span trace_id for opentelemetry log: eb64eefe-3ca6-0f6e-2070-6083162098ea | ||
| 736 | 2026.08.17 23:52:13.418150 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Debug> executeQuery: (from [::1]:56566) SELECT timeseries_key, start_time, timestamp, datum FROM oximeter.measurements_cumulativef64 WHERE timeseries_name = 'virtual_machine:cpu_busy' AND ((greater(timestamp, '2000-01-01 00:00:00.000000000') AND timeseries_key IN (3810615656589673070,5210923722195156639,5419464027902213652,7613129139487427710,11185254949005441257,11873430000991232922,15203798703989247109,15520859351690753739))) ORDER BY timeseries_key, start_time, timestamp LIMIT 1000001 (stage: Complete) | ||
| 737 | 2026.08.17 23:52:13.418585 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 738 | 2026.08.17 23:52:13.418846 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 739 | 2026.08.17 23:52:13.420767 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 1 | ||
| 740 | 2026.08.17 23:52:13.420785 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "(timestamp > '2000-01-01 00:00:00.000000000') AND (timeseries_key IN (3810615656589673070, 5210923722195156639, 5419464027902213652, 7613129139487427710, 11185254949005441257, 11873430000991232922, 15203798703989247109, 15520859351690753739)) AND (timeseries_name = 'virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 741 | 2026.08.17 23:52:13.420935 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64_local | ||
| 742 | 2026.08.17 23:52:13.420983 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 743 | 2026.08.17 23:52:13.421027 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 744 | 2026.08.17 23:52:13.421167 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Key condition: (column 3 in ('946684800', +Inf)), (column 1 in 8-element set), and, (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']), and | ||
| 745 | 2026.08.17 23:52:13.422223 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Used generic exclusion search over index for part all_0_0_0 with 1 steps | ||
| 746 | 2026.08.17 23:52:13.422250 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 747 | 2026.08.17 23:52:13.422255 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Spreading ranges among streams with order | ||
| 748 | 2026.08.17 23:52:13.422351 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 3 steps: "greater(timestamp, '2000-01-01 00:00:00.000000000')", "in(timeseries_key, (3810615656589673070, 5210923722195156639, 5419464027902213652, 7613129139487427710, 11185254949005441257, 11873430000991232922, 15203798703989247109, 15520859351690753739))", "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 749 | 2026.08.17 23:52:13.422356 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 400 rows starting from 0 | ||
| 750 | 2026.08.17 23:52:13.423016 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Debug> executeQuery: Read 400 rows, 25.78 KiB in 0.004981 sec., 80305.15960650472 rows/sec., 5.05 MiB/sec. | ||
| 751 | 2026.08.17 23:52:13.423070 [ 424014 ] {10a7494d-3750-47fe-aef3-bca37c8dae03} <Debug> TCPHandler: Processed in 0.005295604 sec. | ||
| 752 | 2026.08.17 23:52:13.535346 [ 425075 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2360 entries to flush up to offset 2360 | ||
| 753 | 2026.08.17 23:52:13.535355 [ 425074 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 8 | ||
| 754 | 2026.08.17 23:52:13.535399 [ 425073 ] {} <Trace> SystemLog (system.opentelemetry_span_log): Flushing system log, 180 entries to flush up to offset 180 | ||
| 755 | 2026.08.17 23:52:13.535421 [ 425074 ] {} <Debug> SystemLog (system.metric_log): Creating new table system.metric_log for MetricLog | ||
| 756 | 2026.08.17 23:52:13.535416 [ 425075 ] {} <Debug> SystemLog (system.asynchronous_metric_log): Creating new table system.asynchronous_metric_log for AsynchronousMetricLog | ||
| 757 | 2026.08.17 23:52:13.535429 [ 425073 ] {} <Debug> SystemLog (system.opentelemetry_span_log): Creating new table system.opentelemetry_span_log for OpenTelemetrySpanLog | ||
| 758 | 2026.08.17 23:52:13.542300 [ 425075 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Loading data parts | ||
| 759 | 2026.08.17 23:52:13.546557 [ 425075 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): There are no data parts | ||
| 760 | 2026.08.17 23:52:13.542314 [ 425073 ] {} <Debug> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Loading data parts | ||
| 761 | 2026.08.17 23:52:13.547032 [ 425073 ] {} <Debug> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): There are no data parts | ||
| 762 | 2026.08.17 23:52:13.547294 [ 425074 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Loading data parts | ||
| 763 | 2026.08.17 23:52:13.547426 [ 425074 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): There are no data parts | ||
| 764 | 2026.08.17 23:52:13.554443 [ 425073 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 765 | 2026.08.17 23:52:13.554462 [ 425073 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.95 GiB. | ||
| 766 | 2026.08.17 23:52:13.554792 [ 425073 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_1_1_0 (state Temporary) | ||
| 767 | 2026.08.17 23:52:13.554859 [ 425075 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 768 | 2026.08.17 23:52:13.554874 [ 425075 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.95 GiB. | ||
| 769 | 2026.08.17 23:52:13.554987 [ 425073 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Renaming temporary part tmp_insert_202608_1_1_0 to 202608_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 770 | 2026.08.17 23:52:13.557219 [ 425073 ] {} <Trace> SystemLog (system.opentelemetry_span_log): Flushed system log up to offset 180 | ||
| 771 | 2026.08.17 23:52:13.557610 [ 425075 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_1_1_0 (state Temporary) | ||
| 772 | 2026.08.17 23:52:13.560084 [ 425075 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_1_1_0 to 202608_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 773 | 2026.08.17 23:52:13.560250 [ 425075 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 2360 | ||
| 774 | 2026.08.17 23:52:13.564921 [ 425074 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 775 | 2026.08.17 23:52:13.564938 [ 425074 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.95 GiB. | ||
| 776 | 2026.08.17 23:52:13.570663 [ 425074 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_1_1_0 (state Temporary) | ||
| 777 | 2026.08.17 23:52:13.574794 [ 425074 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_1_1_0 to 202608_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 778 | 2026.08.17 23:52:13.575848 [ 425074 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 8 | ||
| 779 | 2026.08.17 23:52:13.722321 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 780 | 2026.08.17 23:52:13.722323 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 781 | 2026.08.17 23:52:13.722416 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=11, queue_size=11, entries=query-0000000000..query-0000000010, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000009, last_skipped_entry_name=none | ||
| 782 | 2026.08.17 23:52:13.722421 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000010 | ||
| 783 | 2026.08.17 23:52:13.722424 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000010 | ||
| 784 | 2026.08.17 23:52:13.722651 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000010 (CREATE DATABASE IF NOT EXISTS oximeter UUID '4be0c896-7297-443c-9ea9-578cef50f5d1') | ||
| 785 | 2026.08.17 23:52:13.731944 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE DATABASE IF NOT EXISTS oximeter UUID '4be0c896-7297-443c-9ea9-578cef50f5d1' | ||
| 786 | 2026.08.17 23:52:13.732000 [ 425097 ] {cec3274d-6f5c-4124-bfa8-90ac6ec09b33} <Trace> executeQuery: Query span trace_id for opentelemetry log: d2d214aa-ea9c-24a7-2cf8-c8dffcee6a39 | ||
| 787 | 2026.08.17 23:52:13.732040 [ 425097 ] {cec3274d-6f5c-4124-bfa8-90ac6ec09b33} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: f8739398-52c9-460e-ba56-ea94cd24a5b9) /* ddl_entry=query-0000000010 */ CREATE DATABASE IF NOT EXISTS oximeter UUID '4be0c896-7297-443c-9ea9-578cef50f5d1' (stage: Complete) | ||
| 788 | 2026.08.17 23:52:13.732124 [ 425097 ] {cec3274d-6f5c-4124-bfa8-90ac6ec09b33} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000010 */ CREATE DATABASE IF NOT EXISTS oximeter UUID '4be0c896-7297-443c-9ea9-578cef50f5d1' | ||
| 789 | 2026.08.17 23:52:13.741098 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 790 | 2026.08.17 23:52:13.846280 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 791 | 2026.08.17 23:52:13.846283 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 792 | 2026.08.17 23:52:13.846371 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=12, queue_size=12, entries=query-0000000000..query-0000000011, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000010, last_skipped_entry_name=none | ||
| 793 | 2026.08.17 23:52:13.846375 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000011 | ||
| 794 | 2026.08.17 23:52:13.846378 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000011 | ||
| 795 | 2026.08.17 23:52:13.846740 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000011 (CREATE TABLE IF NOT EXISTS oximeter.version UUID 'c0ee7deb-8ff1-49dc-a4f8-eaaa2e6fe247' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp)) | ||
| 796 | 2026.08.17 23:52:13.852549 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.version UUID 'c0ee7deb-8ff1-49dc-a4f8-eaaa2e6fe247' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) | ||
| 797 | 2026.08.17 23:52:13.852602 [ 425097 ] {99921961-8f1c-459f-8fe1-4cc45a41e901} <Trace> executeQuery: Query span trace_id for opentelemetry log: 16854907-df7a-03b7-60d1-9a4d8c75a5cc | ||
| 798 | 2026.08.17 23:52:13.852681 [ 425097 ] {99921961-8f1c-459f-8fe1-4cc45a41e901} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 5d992474-32bf-4d4e-a55b-cdee57c21e9d) /* ddl_entry=query-0000000011 */ CREATE TABLE IF NOT EXISTS oximeter.version UUID 'c0ee7deb-8ff1-49dc-a4f8-eaaa2e6fe247' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) (stage: Complete) | ||
| 799 | 2026.08.17 23:52:13.852800 [ 425097 ] {99921961-8f1c-459f-8fe1-4cc45a41e901} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000011 */ CREATE TABLE IF NOT EXISTS oximeter.version UUID 'c0ee7deb-8ff1-49dc-a4f8-eaaa2e6fe247' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) | ||
| 800 | 2026.08.17 23:52:13.861100 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 801 | 2026.08.17 23:52:13.962204 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 802 | 2026.08.17 23:52:13.962207 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 803 | 2026.08.17 23:52:13.962298 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=13, queue_size=13, entries=query-0000000000..query-0000000012, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000011, last_skipped_entry_name=none | ||
| 804 | 2026.08.17 23:52:13.962302 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000012 | ||
| 805 | 2026.08.17 23:52:13.962304 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000012 | ||
| 806 | 2026.08.17 23:52:13.962555 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000012 (CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID 'b3621814-53ee-4a40-975e-9577a7c29267' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30)) | ||
| 807 | 2026.08.17 23:52:13.970715 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID 'b3621814-53ee-4a40-975e-9577a7c29267' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30) | ||
| 808 | 2026.08.17 23:52:13.970763 [ 425097 ] {8fc0801c-d2ca-4034-ae03-418e37957b7b} <Trace> executeQuery: Query span trace_id for opentelemetry log: ca5c133b-54d4-1d93-2968-50c3c57f03ae | ||
| 809 | 2026.08.17 23:52:13.970841 [ 425097 ] {8fc0801c-d2ca-4034-ae03-418e37957b7b} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: df35bc16-dd43-46f9-b3a0-e2346a800b32) /* ddl_entry=query-0000000012 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID 'b3621814-53ee-4a40-975e-9577a7c29267' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30) (stage: Complete) | ||
| 810 | 2026.08.17 23:52:13.970957 [ 425097 ] {8fc0801c-d2ca-4034-ae03-418e37957b7b} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000012 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID 'b3621814-53ee-4a40-975e-9577a7c29267' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30) | ||
| 811 | 2026.08.17 23:52:13.978357 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 812 | 2026.08.17 23:52:14.001603 [ 425094 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 386.02 MiB, peak 394.85 MiB, free memory in arenas 37.78 MiB, will set to 394.37 MiB (RSS), difference: 8.35 MiB | ||
| 813 | 2026.08.17 23:52:14.048413 [ 425093 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' | ||
| 814 | 2026.08.17 23:52:14.048421 [ 425095 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' | ||
| 815 | 2026.08.17 23:52:14.050133 [ 425093 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 816 | 2026.08.17 23:52:14.050143 [ 425095 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 817 | 2026.08.17 23:52:14.050435 [ 425093 ] {} <Information> Application: Setting max_server_memory_usage was set to 27.67 GiB (30.74 GiB available * 0.90 max_server_memory_usage_to_ram_ratio) | ||
| 818 | 2026.08.17 23:52:14.050439 [ 425093 ] {} <Information> Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.37 GiB (30.74 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) | ||
| 819 | 2026.08.17 23:52:14.050442 [ 425093 ] {} <Information> Application: Merges and mutations memory limit is set to 15.37 GiB | ||
| 820 | 2026.08.17 23:52:14.050442 [ 425095 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 821 | 2026.08.17 23:52:14.050614 [ 425093 ] {} <Information> MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (32) for MergeMutateExecutor | ||
| 822 | 2026.08.17 23:52:14.050625 [ 425093 ] {} <Information> MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for MoveExecutor | ||
| 823 | 2026.08.17 23:52:14.050627 [ 425093 ] {} <Information> MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for FetchExecutor | ||
| 824 | 2026.08.17 23:52:14.050629 [ 425093 ] {} <Information> MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for CommonExecutor | ||
| 825 | 2026.08.17 23:52:14.050677 [ 425093 ] {} <Trace> StoragePolicy (default): Storage policy default created, total volumes 1 | ||
| 826 | 2026.08.17 23:52:14.050683 [ 425093 ] {} <Trace> StoragePolicy (default): Storage policy default created, total volumes 1 | ||
| 827 | 2026.08.17 23:52:14.050793 [ 425093 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. | ||
| 828 | 2026.08.17 23:52:14.050942 [ 425093 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 829 | 2026.08.17 23:52:14.084723 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 830 | 2026.08.17 23:52:14.084744 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 831 | 2026.08.17 23:52:14.084849 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=14, queue_size=14, entries=query-0000000000..query-0000000013, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000012, last_skipped_entry_name=none | ||
| 832 | 2026.08.17 23:52:14.084852 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000013 | ||
| 833 | 2026.08.17 23:52:14.084855 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000013 | ||
| 834 | 2026.08.17 23:52:14.085103 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000013 (CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '687a1d0a-fc9f-4251-b55d-83fb6471cb02' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 835 | 2026.08.17 23:52:14.090964 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '687a1d0a-fc9f-4251-b55d-83fb6471cb02' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 836 | 2026.08.17 23:52:14.091007 [ 425097 ] {30e59560-a108-46bf-8e54-9d1b7a3929c0} <Trace> executeQuery: Query span trace_id for opentelemetry log: 478a4286-613c-2c39-95ef-71bd587420d6 | ||
| 837 | 2026.08.17 23:52:14.091067 [ 425097 ] {30e59560-a108-46bf-8e54-9d1b7a3929c0} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: badcb7d2-1906-436a-8baf-6f29321e2ff8) /* ddl_entry=query-0000000013 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '687a1d0a-fc9f-4251-b55d-83fb6471cb02' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 838 | 2026.08.17 23:52:14.091192 [ 425097 ] {30e59560-a108-46bf-8e54-9d1b7a3929c0} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000013 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '687a1d0a-fc9f-4251-b55d-83fb6471cb02' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 839 | 2026.08.17 23:52:14.099688 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 840 | 2026.08.17 23:52:14.204954 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 841 | 2026.08.17 23:52:14.204957 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 842 | 2026.08.17 23:52:14.206290 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=15, queue_size=15, entries=query-0000000000..query-0000000014, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000013, last_skipped_entry_name=none | ||
| 843 | 2026.08.17 23:52:14.206294 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000014 | ||
| 844 | 2026.08.17 23:52:14.206300 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000014 | ||
| 845 | 2026.08.17 23:52:14.206641 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000014 (CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '4ce85b75-228a-434b-bf0a-ac32163275d8' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30)) | ||
| 846 | 2026.08.17 23:52:14.213325 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '4ce85b75-228a-434b-bf0a-ac32163275d8' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 847 | 2026.08.17 23:52:14.213369 [ 425097 ] {3457f23b-4110-4b76-a382-f310e909d71a} <Trace> executeQuery: Query span trace_id for opentelemetry log: 54c3773d-b913-03fd-a8bf-ced8aee8a445 | ||
| 848 | 2026.08.17 23:52:14.213462 [ 425097 ] {3457f23b-4110-4b76-a382-f310e909d71a} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 4b23c3b7-0522-419a-9a03-9c6550868030) /* ddl_entry=query-0000000014 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '4ce85b75-228a-434b-bf0a-ac32163275d8' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) (stage: Complete) | ||
| 849 | 2026.08.17 23:52:14.213643 [ 425097 ] {3457f23b-4110-4b76-a382-f310e909d71a} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000014 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '4ce85b75-228a-434b-bf0a-ac32163275d8' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 850 | 2026.08.17 23:52:14.222684 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 851 | 2026.08.17 23:52:14.329641 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 852 | 2026.08.17 23:52:14.329644 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 853 | 2026.08.17 23:52:14.329732 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=16, queue_size=16, entries=query-0000000000..query-0000000015, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000014, last_skipped_entry_name=none | ||
| 854 | 2026.08.17 23:52:14.329736 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000015 | ||
| 855 | 2026.08.17 23:52:14.329739 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000015 | ||
| 856 | 2026.08.17 23:52:14.329996 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000015 (CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID 'a46085f8-d016-49d0-8bec-67bf5913f41d' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 857 | 2026.08.17 23:52:14.337077 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID 'a46085f8-d016-49d0-8bec-67bf5913f41d' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 858 | 2026.08.17 23:52:14.337120 [ 425097 ] {74a66902-f2f9-4c2f-8c14-387a9073bf59} <Trace> executeQuery: Query span trace_id for opentelemetry log: c1c60bc6-4a1b-9316-0669-ee0234f1dcc2 | ||
| 859 | 2026.08.17 23:52:14.337171 [ 425097 ] {74a66902-f2f9-4c2f-8c14-387a9073bf59} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 0d6b380e-6b10-4314-998b-91b7ea50b1ba) /* ddl_entry=query-0000000015 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID 'a46085f8-d016-49d0-8bec-67bf5913f41d' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 860 | 2026.08.17 23:52:14.337353 [ 425097 ] {74a66902-f2f9-4c2f-8c14-387a9073bf59} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000015 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID 'a46085f8-d016-49d0-8bec-67bf5913f41d' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 861 | 2026.08.17 23:52:14.340791 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 862 | 2026.08.17 23:52:14.445957 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 863 | 2026.08.17 23:52:14.445960 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 864 | 2026.08.17 23:52:14.446028 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=17, queue_size=17, entries=query-0000000000..query-0000000016, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000015, last_skipped_entry_name=none | ||
| 865 | 2026.08.17 23:52:14.446033 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000016 | ||
| 866 | 2026.08.17 23:52:14.446035 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000016 | ||
| 867 | 2026.08.17 23:52:14.446307 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000016 (CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID 'a6697841-90b5-4b0b-8f35-655266f19b34' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30)) | ||
| 868 | 2026.08.17 23:52:14.454567 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID 'a6697841-90b5-4b0b-8f35-655266f19b34' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 869 | 2026.08.17 23:52:14.454616 [ 425097 ] {04400cbc-7efc-4c20-af0c-0897cf41ce75} <Trace> executeQuery: Query span trace_id for opentelemetry log: 3fcfcacc-1cd9-ba4d-63b7-c3e3436382a9 | ||
| 870 | 2026.08.17 23:52:14.454693 [ 425097 ] {04400cbc-7efc-4c20-af0c-0897cf41ce75} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: b57a40e3-8d42-40b9-aed2-8491ccacdf96) /* ddl_entry=query-0000000016 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID 'a6697841-90b5-4b0b-8f35-655266f19b34' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) (stage: Complete) | ||
| 871 | 2026.08.17 23:52:14.454886 [ 425097 ] {04400cbc-7efc-4c20-af0c-0897cf41ce75} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000016 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID 'a6697841-90b5-4b0b-8f35-655266f19b34' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 872 | 2026.08.17 23:52:14.463786 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 873 | 2026.08.17 23:52:14.533945 [ 431323 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47252, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 874 | 2026.08.17 23:52:14.533998 [ 431323 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_uuid_local%2Freplicas%2F1&part=all_0_0_0&client_protocol_version=8&compress=false | ||
| 875 | 2026.08.17 23:52:14.534009 [ 431323 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Replicated PartsService): Sending part all_0_0_0 | ||
| 876 | 2026.08.17 23:52:14.534213 [ 431323 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 877 | 2026.08.17 23:52:14.571649 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 878 | 2026.08.17 23:52:14.571654 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 879 | 2026.08.17 23:52:14.571753 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=18, queue_size=18, entries=query-0000000000..query-0000000017, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000016, last_skipped_entry_name=none | ||
| 880 | 2026.08.17 23:52:14.571757 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000017 | ||
| 881 | 2026.08.17 23:52:14.571760 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000017 | ||
| 882 | 2026.08.17 23:52:14.572038 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000017 (CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '3a17e6d0-bd01-498b-ad0e-e9245d564d41' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 883 | 2026.08.17 23:52:14.579277 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '3a17e6d0-bd01-498b-ad0e-e9245d564d41' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 884 | 2026.08.17 23:52:14.579330 [ 425097 ] {22eec810-41c9-4c95-bf95-d17cba5d7fb6} <Trace> executeQuery: Query span trace_id for opentelemetry log: 946a9ab4-02ea-e5b8-a3b3-83c3d60f5bb6 | ||
| 885 | 2026.08.17 23:52:14.579416 [ 425097 ] {22eec810-41c9-4c95-bf95-d17cba5d7fb6} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 7eca9f79-4231-498f-aced-aa2210a52bc5) /* ddl_entry=query-0000000017 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '3a17e6d0-bd01-498b-ad0e-e9245d564d41' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 886 | 2026.08.17 23:52:14.579640 [ 425097 ] {22eec810-41c9-4c95-bf95-d17cba5d7fb6} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000017 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '3a17e6d0-bd01-498b-ad0e-e9245d564d41' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 887 | 2026.08.17 23:52:14.588720 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 888 | 2026.08.17 23:52:14.694376 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 889 | 2026.08.17 23:52:14.694374 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 890 | 2026.08.17 23:52:14.695628 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=19, queue_size=19, entries=query-0000000000..query-0000000018, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000017, last_skipped_entry_name=none | ||
| 891 | 2026.08.17 23:52:14.696988 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000018 | ||
| 892 | 2026.08.17 23:52:14.696991 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000018 | ||
| 893 | 2026.08.17 23:52:14.697328 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000018 (CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID 'fedf803c-4b42-4f88-bda5-0cf52392d08b' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name)) | ||
| 894 | 2026.08.17 23:52:14.702683 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID 'fedf803c-4b42-4f88-bda5-0cf52392d08b' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) | ||
| 895 | 2026.08.17 23:52:14.702741 [ 425097 ] {b96daee1-cf4a-4923-aa74-a8bd8380b35f} <Trace> executeQuery: Query span trace_id for opentelemetry log: ddc37f34-a0b7-c7f7-47c9-c53e276b6cba | ||
| 896 | 2026.08.17 23:52:14.702846 [ 425097 ] {b96daee1-cf4a-4923-aa74-a8bd8380b35f} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 5380d490-e09b-4ff0-aed1-d03fc091bde8) /* ddl_entry=query-0000000018 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID 'fedf803c-4b42-4f88-bda5-0cf52392d08b' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) (stage: Complete) | ||
| 897 | 2026.08.17 23:52:14.703051 [ 425097 ] {b96daee1-cf4a-4923-aa74-a8bd8380b35f} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000018 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID 'fedf803c-4b42-4f88-bda5-0cf52392d08b' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) | ||
| 898 | 2026.08.17 23:52:14.711789 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 899 | 2026.08.17 23:52:14.817840 [ 425097 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 900 | 2026.08.17 23:52:14.817845 [ 425098 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 901 | 2026.08.17 23:52:14.817965 [ 425097 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=20, queue_size=20, entries=query-0000000000..query-0000000019, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000018, last_skipped_entry_name=none | ||
| 902 | 2026.08.17 23:52:14.817974 [ 425097 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000019 | ||
| 903 | 2026.08.17 23:52:14.817978 [ 425097 ] {} <Trace> DDLWorker: Checking task query-0000000019 | ||
| 904 | 2026.08.17 23:52:14.818417 [ 425097 ] {} <Debug> DDLWorker: Processing task query-0000000019 (CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID '1cc18418-904a-4736-9734-d5efdce75180' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 905 | 2026.08.17 23:52:14.826590 [ 425097 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID '1cc18418-904a-4736-9734-d5efdce75180' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 906 | 2026.08.17 23:52:14.826660 [ 425097 ] {4d70a152-f492-4635-86ee-116d95163518} <Trace> executeQuery: Query span trace_id for opentelemetry log: beac905f-b105-04bb-1fc3-91b58bf7f6ea | ||
| 907 | 2026.08.17 23:52:14.826806 [ 425097 ] {4d70a152-f492-4635-86ee-116d95163518} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: a500356f-414f-4204-8145-0b8bce4ec994) /* ddl_entry=query-0000000019 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID '1cc18418-904a-4736-9734-d5efdce75180' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 908 | 2026.08.17 23:52:14.827032 [ 425097 ] {4d70a152-f492-4635-86ee-116d95163518} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000019 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID '1cc18418-904a-4736-9734-d5efdce75180' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 909 | 2026.08.17 23:52:14.835601 [ 425097 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 910 | 2026.08.17 23:52:15.168105 [ 431888 ] {} <Information> Application: Will watch for the process with pid 431908 | ||
| 911 | 2026.08.17 23:52:15.168175 [ 431908 ] {} <Information> Application: Forked a child process to watch | ||
| 912 | 2026.08.17 23:52:15.168345 [ 431908 ] {} <Information> SentryWriter: Sending crash reports is disabled | ||
| 913 | 2026.08.17 23:52:15.168427 [ 431908 ] {} <Trace> Pipe: Pipe capacity is 1.00 MiB | ||
| 914 | 2026.08.17 23:52:15.198123 [ 431908 ] {} <Information> Application: Starting ClickHouse 23.8.7.24 (revision: 54477, git hash: 812b95e14ba8cf744bf1d70e6de607cf130a79fa, build id: 6484B75ECB16F4FF87A88E10249A45E1BDBDE2E5), PID 431908 | ||
| 915 | 2026.08.17 23:52:15.198595 [ 431908 ] {} <Information> Application: starting up | ||
| 916 | 2026.08.17 23:52:15.198613 [ 431908 ] {} <Information> Application: OS name: Linux, version: 5.15.0-1015-aws, architecture: x86_64 | ||
| 917 | 2026.08.17 23:52:15.201112 [ 431908 ] {} <Information> Application: Available RAM: 30.74 GiB; physical cores: 16; logical cores: 16. | ||
| 918 | 2026.08.17 23:52:15.202187 [ 431908 ] {} <Trace> AsynchronousMetrics: Scanning /sys/class/thermal | ||
| 919 | 2026.08.17 23:52:15.202196 [ 431908 ] {} <Trace> AsynchronousMetrics: Scanning /sys/block | ||
| 920 | 2026.08.17 23:52:15.202227 [ 431908 ] {} <Trace> AsynchronousMetrics: Scanning /sys/devices/system/edac | ||
| 921 | 2026.08.17 23:52:15.202234 [ 431908 ] {} <Trace> AsynchronousMetrics: Scanning /sys/class/hwmon | ||
| 922 | 2026.08.17 23:52:15.305589 [ 431908 ] {} <Information> Application: Integrity check of the executable successfully passed (checksum: 63E397DB90569A93F13765E06F4A1FBF) | ||
| 923 | 2026.08.17 23:52:15.305650 [ 431908 ] {} <Information> StatusFile: Status file /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/data/status already exists - unclean restart. Contents: | ||
| 924 | PID: 423933 | ||
| 925 | Started at: 2026-08-17 23:52:05 | ||
| 926 | Revision: 54477 | ||
| 927 | |||
| 928 | 2026.08.17 23:52:15.305719 [ 431908 ] {} <Debug> Application: Set max number of file descriptors to 524288 (was 1024). | ||
| 929 | 2026.08.17 23:52:15.305722 [ 431908 ] {} <Debug> Application: rlimit on number of threads is 125877 | ||
| 930 | 2026.08.17 23:52:15.305726 [ 431908 ] {} <Debug> Application: Initializing DateLUT. | ||
| 931 | 2026.08.17 23:52:15.305728 [ 431908 ] {} <Trace> Application: Initialized DateLUT with time zone 'UTC'. | ||
| 932 | 2026.08.17 23:52:15.305737 [ 431908 ] {} <Debug> Context: Setting up /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/data/tmp/ to store temporary data in it | ||
| 933 | 2026.08.17 23:52:15.305775 [ 431908 ] {} <Debug> Application: Initializing interserver credentials. | ||
| 934 | 2026.08.17 23:52:15.305961 [ 431908 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' | ||
| 935 | 2026.08.17 23:52:15.306368 [ 431908 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 936 | 2026.08.17 23:52:15.306668 [ 431908 ] {} <Information> Application: Setting max_server_memory_usage was set to 27.67 GiB (30.74 GiB available * 0.90 max_server_memory_usage_to_ram_ratio) | ||
| 937 | 2026.08.17 23:52:15.308443 [ 431908 ] {} <Information> Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.37 GiB (30.74 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) | ||
| 938 | 2026.08.17 23:52:15.308453 [ 431908 ] {} <Information> Application: Merges and mutations memory limit is set to 15.37 GiB | ||
| 939 | 2026.08.17 23:52:15.308696 [ 431908 ] {} <Information> BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 940 | 2026.08.17 23:52:15.309070 [ 431908 ] {} <Information> BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 128 threads | ||
| 941 | 2026.08.17 23:52:15.316823 [ 431908 ] {} <Information> BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 942 | 2026.08.17 23:52:15.319034 [ 431908 ] {} <Information> BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 943 | 2026.08.17 23:52:15.320124 [ 431908 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. | ||
| 944 | 2026.08.17 23:52:15.320139 [ 431908 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 945 | 2026.08.17 23:52:15.320352 [ 431908 ] {} <Information> Application: Listening for replica communication (interserver): http://[::1]:19401 | ||
| 946 | 2026.08.17 23:52:15.321320 [ 431908 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' | ||
| 947 | 2026.08.17 23:52:15.322025 [ 431908 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 948 | 2026.08.17 23:52:15.322529 [ 431908 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 949 | 2026.08.17 23:52:15.322653 [ 431908 ] {} <Debug> Access(user directories): Added users_xml access storage 'users_xml', path: /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml | ||
| 950 | 2026.08.17 23:52:15.323004 [ 431908 ] {} <Information> Context: Initialized background executor for merges and mutations with num_threads=16, num_tasks=32, scheduling_policy=round_robin | ||
| 951 | 2026.08.17 23:52:15.323217 [ 431908 ] {} <Information> Context: Initialized background executor for move operations with num_threads=8, num_tasks=8 | ||
| 952 | 2026.08.17 23:52:15.323411 [ 431908 ] {} <Information> Context: Initialized background executor for fetches with num_threads=8, num_tasks=8 | ||
| 953 | 2026.08.17 23:52:15.324993 [ 431908 ] {} <Information> Context: Initialized background executor for common operations (e.g. clearing old parts) with num_threads=8, num_tasks=8 | ||
| 954 | 2026.08.17 23:52:15.325367 [ 431908 ] {} <Information> DNSCacheUpdater: Update period 15 seconds | ||
| 955 | 2026.08.17 23:52:15.325403 [ 431908 ] {} <Information> Application: Loading metadata from /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/data/ | ||
| 956 | 2026.08.17 23:52:15.325418 [ 432028 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 957 | 2026.08.17 23:52:15.325477 [ 432028 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 958 | 2026.08.17 23:52:15.329997 [ 431908 ] {} <Information> DatabaseAtomic (system): Metadata processed, database system has 3 tables and 0 dictionaries in total. | ||
| 959 | 2026.08.17 23:52:15.330020 [ 431908 ] {} <Information> TablesLoader: Parsed metadata of 3 tables in 1 databases in 0.004314113 sec | ||
| 960 | 2026.08.17 23:52:15.330399 [ 431908 ] {} <Trace> ReferentialDeps: No tables | ||
| 961 | 2026.08.17 23:52:15.330407 [ 431908 ] {} <Trace> LoadingDeps: Table system.opentelemetry_span_log has no dependencies (level 0) | ||
| 962 | 2026.08.17 23:52:15.330409 [ 431908 ] {} <Trace> LoadingDeps: Table system.metric_log has no dependencies (level 0) | ||
| 963 | 2026.08.17 23:52:15.330411 [ 431908 ] {} <Trace> LoadingDeps: Table system.asynchronous_metric_log has no dependencies (level 0) | ||
| 964 | 2026.08.17 23:52:15.330437 [ 431908 ] {} <Information> TablesLoader: Loading 3 tables with dependency level 0 | ||
| 965 | 2026.08.17 23:52:15.331356 [ 432254 ] {} <Trace> StoragePolicy (default): Storage policy default created, total volumes 1 | ||
| 966 | 2026.08.17 23:52:15.331404 [ 432254 ] {} <Debug> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Loading data parts | ||
| 967 | 2026.08.17 23:52:15.331571 [ 432255 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Loading data parts | ||
| 968 | 2026.08.17 23:52:15.331628 [ 432254 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Will load 1 parts using up to 64 threads | ||
| 969 | 2026.08.17 23:52:15.331649 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Will load 1 parts using up to 64 threads | ||
| 970 | 2026.08.17 23:52:15.331649 [ 432257 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Loading Active part 202608_1_1_0 from disk default | ||
| 971 | 2026.08.17 23:52:15.333106 [ 432258 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Loading Active part 202608_1_1_0 from disk default | ||
| 972 | 2026.08.17 23:52:15.333423 [ 432258 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Finished loading Active part 202608_1_1_0 on disk default | ||
| 973 | 2026.08.17 23:52:15.333436 [ 432257 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Finished loading Active part 202608_1_1_0 on disk default | ||
| 974 | 2026.08.17 23:52:15.333619 [ 432255 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Loaded data parts (1 items) | ||
| 975 | 2026.08.17 23:52:15.333631 [ 432254 ] {} <Debug> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Loaded data parts (1 items) | ||
| 976 | 2026.08.17 23:52:15.333830 [ 432256 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Loading data parts | ||
| 977 | 2026.08.17 23:52:15.333888 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Will load 1 parts using up to 64 threads | ||
| 978 | 2026.08.17 23:52:15.333929 [ 432259 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Loading Active part 202608_1_1_0 from disk default | ||
| 979 | 2026.08.17 23:52:15.338693 [ 432259 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Finished loading Active part 202608_1_1_0 on disk default | ||
| 980 | 2026.08.17 23:52:15.338814 [ 432256 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Loaded data parts (1 items) | ||
| 981 | 2026.08.17 23:52:15.339153 [ 431908 ] {} <Information> DatabaseAtomic (system): Starting up tables. | ||
| 982 | 2026.08.17 23:52:15.340689 [ 431908 ] {} <Debug> SystemLog: Not creating system.query_log since corresponding section 'query_log' is missing from config | ||
| 983 | 2026.08.17 23:52:15.340699 [ 431908 ] {} <Debug> SystemLog: Not creating system.query_thread_log since corresponding section 'query_thread_log' is missing from config | ||
| 984 | 2026.08.17 23:52:15.340703 [ 431908 ] {} <Debug> SystemLog: Not creating system.part_log since corresponding section 'part_log' is missing from config | ||
| 985 | 2026.08.17 23:52:15.340707 [ 431908 ] {} <Debug> SystemLog: Not creating system.trace_log since corresponding section 'trace_log' is missing from config | ||
| 986 | 2026.08.17 23:52:15.340710 [ 431908 ] {} <Debug> SystemLog: Not creating system.crash_log since corresponding section 'crash_log' is missing from config | ||
| 987 | 2026.08.17 23:52:15.340714 [ 431908 ] {} <Debug> SystemLog: Not creating system.text_log since corresponding section 'text_log' is missing from config | ||
| 988 | 2026.08.17 23:52:15.340718 [ 431908 ] {} <Debug> SystemLog: Creating system.metric_log from metric_log | ||
| 989 | 2026.08.17 23:52:15.341738 [ 431908 ] {} <Debug> SystemLog: Not creating system.filesystem_cache_log since corresponding section 'filesystem_cache_log' is missing from config | ||
| 990 | 2026.08.17 23:52:15.343195 [ 431908 ] {} <Debug> SystemLog: Not creating system.filesystem_read_prefetches_log since corresponding section 'filesystem_read_prefetches_log' is missing from config | ||
| 991 | 2026.08.17 23:52:15.343202 [ 431908 ] {} <Debug> SystemLog: Creating system.asynchronous_metric_log from asynchronous_metric_log | ||
| 992 | 2026.08.17 23:52:15.343289 [ 431908 ] {} <Debug> SystemLog: Creating system.opentelemetry_span_log from opentelemetry_span_log | ||
| 993 | 2026.08.17 23:52:15.343385 [ 431908 ] {} <Debug> SystemLog: Not creating system.query_views_log since corresponding section 'query_views_log' is missing from config | ||
| 994 | 2026.08.17 23:52:15.343389 [ 431908 ] {} <Debug> SystemLog: Not creating system.zookeeper_log since corresponding section 'zookeeper_log' is missing from config | ||
| 995 | 2026.08.17 23:52:15.343392 [ 431908 ] {} <Debug> SystemLog: Not creating system.session_log since corresponding section 'session_log' is missing from config | ||
| 996 | 2026.08.17 23:52:15.343396 [ 431908 ] {} <Debug> SystemLog: Not creating system.transactions_info_log since corresponding section 'transactions_info_log' is missing from config | ||
| 997 | 2026.08.17 23:52:15.343403 [ 431908 ] {} <Debug> SystemLog: Not creating system.processors_profile_log since corresponding section 'processors_profile_log' is missing from config | ||
| 998 | 2026.08.17 23:52:15.343407 [ 431908 ] {} <Debug> SystemLog: Not creating system.asynchronous_insert_log since corresponding section 'asynchronous_insert_log' is missing from config | ||
| 999 | 2026.08.17 23:52:15.348995 [ 431908 ] {} <Information> DatabaseCatalog: Found 0 partially dropped tables. Will load them and retry removal. | ||
| 1000 | 2026.08.17 23:52:15.349460 [ 431908 ] {} <Information> DatabaseAtomic (default): Metadata processed, database default has 0 tables and 0 dictionaries in total. | ||
| 1001 | 2026.08.17 23:52:15.350350 [ 431908 ] {} <Information> DatabaseAtomic (oximeter): Metadata processed, database oximeter has 9 tables and 0 dictionaries in total. | ||
| 1002 | 2026.08.17 23:52:15.350359 [ 431908 ] {} <Information> TablesLoader: Parsed metadata of 9 tables in 2 databases in 0.000906341 sec | ||
| 1003 | 2026.08.17 23:52:15.350428 [ 431908 ] {} <Trace> ReferentialDeps: Table oximeter.timeseries_schema_local has no dependencies (level 0) | ||
| 1004 | 2026.08.17 23:52:15.350430 [ 431908 ] {} <Trace> ReferentialDeps: Table oximeter.fields_i64_local has no dependencies (level 0) | ||
| 1005 | 2026.08.17 23:52:15.350433 [ 431908 ] {} <Trace> ReferentialDeps: Table oximeter.measurements_cumulativef64_local has no dependencies (level 0) | ||
| 1006 | 2026.08.17 23:52:15.350434 [ 431908 ] {} <Trace> ReferentialDeps: Table oximeter.fields_uuid_local has no dependencies (level 0) | ||
| 1007 | 2026.08.17 23:52:15.350437 [ 431908 ] {} <Trace> ReferentialDeps: Table oximeter.timeseries_schema has 1 dependencies: oximeter.timeseries_schema_local (level 1) | ||
| 1008 | 2026.08.17 23:52:15.350439 [ 431908 ] {} <Trace> ReferentialDeps: Table oximeter.fields_i64 has 1 dependencies: oximeter.fields_i64_local (level 1) | ||
| 1009 | 2026.08.17 23:52:15.350441 [ 431908 ] {} <Trace> ReferentialDeps: Table oximeter.measurements_cumulativef64 has 1 dependencies: oximeter.measurements_cumulativef64_local (level 1) | ||
| 1010 | 2026.08.17 23:52:15.350442 [ 431908 ] {} <Trace> ReferentialDeps: Table oximeter.fields_uuid has 1 dependencies: oximeter.fields_uuid_local (level 1) | ||
| 1011 | 2026.08.17 23:52:15.350444 [ 431908 ] {} <Trace> LoadingDeps: Table oximeter.version has no dependencies (level 0) | ||
| 1012 | 2026.08.17 23:52:15.350446 [ 431908 ] {} <Trace> LoadingDeps: Table oximeter.timeseries_schema has no dependencies (level 0) | ||
| 1013 | 2026.08.17 23:52:15.350448 [ 431908 ] {} <Trace> LoadingDeps: Table oximeter.measurements_cumulativef64_local has no dependencies (level 0) | ||
| 1014 | 2026.08.17 23:52:15.350450 [ 431908 ] {} <Trace> LoadingDeps: Table oximeter.measurements_cumulativef64 has no dependencies (level 0) | ||
| 1015 | 2026.08.17 23:52:15.350451 [ 431908 ] {} <Trace> LoadingDeps: Table oximeter.fields_uuid_local has no dependencies (level 0) | ||
| 1016 | 2026.08.17 23:52:15.350452 [ 431908 ] {} <Trace> LoadingDeps: Table oximeter.fields_uuid has no dependencies (level 0) | ||
| 1017 | 2026.08.17 23:52:15.350454 [ 431908 ] {} <Trace> LoadingDeps: Table oximeter.timeseries_schema_local has no dependencies (level 0) | ||
| 1018 | 2026.08.17 23:52:15.350470 [ 431908 ] {} <Trace> LoadingDeps: Table oximeter.fields_i64_local has no dependencies (level 0) | ||
| 1019 | 2026.08.17 23:52:15.350472 [ 431908 ] {} <Trace> LoadingDeps: Table oximeter.fields_i64 has no dependencies (level 0) | ||
| 1020 | 2026.08.17 23:52:15.350490 [ 431908 ] {} <Information> TablesLoader: Loading 9 tables with dependency level 0 | ||
| 1021 | 2026.08.17 23:52:15.350830 [ 432265 ] {} <Debug> StorageDistributed (measurements_cumulativef64): Auto-increment is 0 | ||
| 1022 | 2026.08.17 23:52:15.350938 [ 432262 ] {} <Debug> StorageDistributed (timeseries_schema): Auto-increment is 0 | ||
| 1023 | 2026.08.17 23:52:15.352650 [ 432264 ] {} <Debug> StorageDistributed (fields_i64): Auto-increment is 0 | ||
| 1024 | 2026.08.17 23:52:15.352385 [ 432259 ] {} <Debug> StorageDistributed (fields_uuid): Auto-increment is 0 | ||
| 1025 | 2026.08.17 23:52:15.356593 [ 432267 ] {} <Information> ZooKeeperClient: Connected to ZooKeeper at [::1]:19002 with session_id 7 | ||
| 1026 | 2026.08.17 23:52:15.356826 [ 432267 ] {} <Information> ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled | ||
| 1027 | 2026.08.17 23:52:15.356835 [ 432267 ] {} <Information> ZooKeeperClient: Keeper feature flag MULTI_READ: enabled | ||
| 1028 | 2026.08.17 23:52:15.356837 [ 432267 ] {} <Information> ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: disabled | ||
| 1029 | 2026.08.17 23:52:15.356841 [ 432267 ] {} <Trace> ZooKeeper: Initialized, hosts: [::1]:19001,[::1]:19002,[::1]:19003 | ||
| 1030 | 2026.08.17 23:52:15.357124 [ 432267 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Loading data parts | ||
| 1031 | 2026.08.17 23:52:15.357138 [ 432266 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Loading data parts | ||
| 1032 | 2026.08.17 23:52:15.357163 [ 432268 ] {} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Loading data parts | ||
| 1033 | 2026.08.17 23:52:15.357187 [ 432263 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Loading data parts | ||
| 1034 | 2026.08.17 23:52:15.357189 [ 432257 ] {} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Loading data parts | ||
| 1035 | 2026.08.17 23:52:15.357386 [ 432266 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Will load 1 parts using up to 64 threads | ||
| 1036 | 2026.08.17 23:52:15.357400 [ 432267 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Will load 1 parts using up to 64 threads | ||
| 1037 | 2026.08.17 23:52:15.357485 [ 432263 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Will load 1 parts using up to 64 threads | ||
| 1038 | 2026.08.17 23:52:15.358741 [ 432270 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Loading Active part all_0_0_0 from disk default | ||
| 1039 | 2026.08.17 23:52:15.357533 [ 432257 ] {} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): There are no data parts | ||
| 1040 | 2026.08.17 23:52:15.358751 [ 432274 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Loading Active part all_0_0_0 from disk default | ||
| 1041 | 2026.08.17 23:52:15.357498 [ 432268 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Will load 1 parts using up to 64 threads | ||
| 1042 | 2026.08.17 23:52:15.358762 [ 432257 ] {} <Information> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Table will be in readonly mode until initialization is finished | ||
| 1043 | 2026.08.17 23:52:15.358776 [ 432273 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Loading Active part all_0_0_0 from disk default | ||
| 1044 | 2026.08.17 23:52:15.358936 [ 432275 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Loading Active part all_0_0_0 from disk default | ||
| 1045 | 2026.08.17 23:52:15.359025 [ 432274 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Finished loading Active part all_0_0_0 on disk default | ||
| 1046 | 2026.08.17 23:52:15.359025 [ 432270 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Finished loading Active part all_0_0_0 on disk default | ||
| 1047 | 2026.08.17 23:52:15.359046 [ 432273 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Finished loading Active part all_0_0_0 on disk default | ||
| 1048 | 2026.08.17 23:52:15.359059 [ 432267 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Loaded data parts (1 items) | ||
| 1049 | 2026.08.17 23:52:15.359065 [ 432267 ] {} <Information> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Table will be in readonly mode until initialization is finished | ||
| 1050 | 2026.08.17 23:52:15.359084 [ 432263 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Loaded data parts (1 items) | ||
| 1051 | 2026.08.17 23:52:15.359090 [ 432263 ] {} <Information> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Table will be in readonly mode until initialization is finished | ||
| 1052 | 2026.08.17 23:52:15.359103 [ 432266 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Loaded data parts (1 items) | ||
| 1053 | 2026.08.17 23:52:15.359110 [ 432266 ] {} <Information> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Table will be in readonly mode until initialization is finished | ||
| 1054 | 2026.08.17 23:52:15.359246 [ 432275 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Finished loading Active part all_0_0_0 on disk default | ||
| 1055 | 2026.08.17 23:52:15.359277 [ 432268 ] {} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Loaded data parts (1 items) | ||
| 1056 | 2026.08.17 23:52:15.359284 [ 432268 ] {} <Information> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Table will be in readonly mode until initialization is finished | ||
| 1057 | 2026.08.17 23:52:15.359329 [ 431908 ] {} <Information> DatabaseAtomic (default): Starting up tables. | ||
| 1058 | 2026.08.17 23:52:15.359333 [ 431908 ] {} <Information> DatabaseAtomic (oximeter): Starting up tables. | ||
| 1059 | 2026.08.17 23:52:15.359346 [ 432264 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Starting up table | ||
| 1060 | 2026.08.17 23:52:15.359352 [ 432257 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Starting up table | ||
| 1061 | 2026.08.17 23:52:15.359353 [ 432262 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Starting up table | ||
| 1062 | 2026.08.17 23:52:15.359355 [ 432259 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Starting up table | ||
| 1063 | 2026.08.17 23:52:15.359346 [ 432265 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Starting up table | ||
| 1064 | 2026.08.17 23:52:15.396367 [ 432091 ] {} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Creating shared ID for table oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4) | ||
| 1065 | 2026.08.17 23:52:15.396385 [ 432058 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Creating shared ID for table oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) | ||
| 1066 | 2026.08.17 23:52:15.396388 [ 432023 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Creating shared ID for table oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) | ||
| 1067 | 2026.08.17 23:52:15.396403 [ 432057 ] {} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Creating shared ID for table oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) | ||
| 1068 | 2026.08.17 23:52:15.402538 [ 432091 ] {} <Debug> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Initializing table shared ID with 157c979f-eec8-4d9d-8340-ef2e89475af4 | ||
| 1069 | 2026.08.17 23:52:15.402542 [ 432058 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Initializing table shared ID with eaa5beef-895d-41af-915b-bd6fb56613c4 | ||
| 1070 | 2026.08.17 23:52:15.402577 [ 432057 ] {} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Initializing table shared ID with 05a0062d-b532-435a-91d7-b9da99374f1d | ||
| 1071 | 2026.08.17 23:52:15.402577 [ 432023 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Initializing table shared ID with 24dea9d7-064d-49c2-9599-9587402bf78f | ||
| 1072 | 2026.08.17 23:52:15.402651 [ 432091 ] {} <Information> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Became leader | ||
| 1073 | 2026.08.17 23:52:15.402654 [ 432091 ] {} <Trace> oximeter.version (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: false | ||
| 1074 | 2026.08.17 23:52:15.402657 [ 432091 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Trying to startup table from right now | ||
| 1075 | 2026.08.17 23:52:15.402659 [ 432091 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 1076 | 2026.08.17 23:52:15.402660 [ 432058 ] {} <Information> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Became leader | ||
| 1077 | 2026.08.17 23:52:15.402662 [ 432091 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1078 | 2026.08.17 23:52:15.402663 [ 432058 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: false | ||
| 1079 | 2026.08.17 23:52:15.402669 [ 432058 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Trying to startup table from right now | ||
| 1080 | 2026.08.17 23:52:15.402671 [ 432058 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 1081 | 2026.08.17 23:52:15.402673 [ 432058 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1082 | 2026.08.17 23:52:15.402698 [ 432057 ] {} <Information> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Became leader | ||
| 1083 | 2026.08.17 23:52:15.402702 [ 432057 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: false | ||
| 1084 | 2026.08.17 23:52:15.402704 [ 432057 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Trying to startup table from right now | ||
| 1085 | 2026.08.17 23:52:15.402705 [ 432023 ] {} <Information> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Became leader | ||
| 1086 | 2026.08.17 23:52:15.402710 [ 432023 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: false | ||
| 1087 | 2026.08.17 23:52:15.402711 [ 432099 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Creating shared ID for table oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) | ||
| 1088 | 2026.08.17 23:52:15.402713 [ 432023 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Trying to startup table from right now | ||
| 1089 | 2026.08.17 23:52:15.402715 [ 432023 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 1090 | 2026.08.17 23:52:15.402717 [ 432023 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1091 | 2026.08.17 23:52:15.402706 [ 432057 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 1092 | 2026.08.17 23:52:15.402724 [ 432057 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1093 | 2026.08.17 23:52:15.402761 [ 432099 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Initializing table shared ID with 2d79911c-8882-4309-a914-7e64c31972ae | ||
| 1094 | 2026.08.17 23:52:15.402852 [ 432099 ] {} <Information> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Became leader | ||
| 1095 | 2026.08.17 23:52:15.402854 [ 432099 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: false | ||
| 1096 | 2026.08.17 23:52:15.402856 [ 432099 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Trying to startup table from right now | ||
| 1097 | 2026.08.17 23:52:15.402858 [ 432099 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Activating replica. | ||
| 1098 | 2026.08.17 23:52:15.402859 [ 432099 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1099 | 2026.08.17 23:52:15.421363 [ 432091 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1100 | 2026.08.17 23:52:15.421364 [ 432058 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1101 | 2026.08.17 23:52:15.421427 [ 432091 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1102 | 2026.08.17 23:52:15.421430 [ 432091 ] {} <Debug> oximeter.version (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/version/replicas/1/queue | ||
| 1103 | 2026.08.17 23:52:15.421432 [ 432058 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1104 | 2026.08.17 23:52:15.421430 [ 432057 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1105 | 2026.08.17 23:52:15.421436 [ 432058 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/fields_uuid_local/replicas/1/queue | ||
| 1106 | 2026.08.17 23:52:15.421453 [ 432023 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1107 | 2026.08.17 23:52:15.424310 [ 432091 ] {} <Debug> oximeter.version (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1108 | 2026.08.17 23:52:15.424322 [ 432057 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1109 | 2026.08.17 23:52:15.424328 [ 432057 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/timeseries_schema_local/replicas/1/queue | ||
| 1110 | 2026.08.17 23:52:15.424343 [ 432023 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1111 | 2026.08.17 23:52:15.424344 [ 432058 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1112 | 2026.08.17 23:52:15.424346 [ 432023 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/measurements_cumulativef64_local/replicas/1/queue | ||
| 1113 | 2026.08.17 23:52:15.424413 [ 432091 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1114 | 2026.08.17 23:52:15.427497 [ 432057 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1115 | 2026.08.17 23:52:15.427500 [ 432023 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1116 | 2026.08.17 23:52:15.427498 [ 432058 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1117 | 2026.08.17 23:52:15.427554 [ 432057 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1118 | 2026.08.17 23:52:15.427560 [ 432023 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1119 | 2026.08.17 23:52:15.427563 [ 432099 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1120 | 2026.08.17 23:52:15.427620 [ 432099 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1121 | 2026.08.17 23:52:15.427623 [ 432099 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/fields_i64_local/replicas/1/queue | ||
| 1122 | 2026.08.17 23:52:15.427685 [ 432099 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1123 | 2026.08.17 23:52:15.427732 [ 432099 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1124 | 2026.08.17 23:52:15.427808 [ 432058 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000001 - log-0000000001 | ||
| 1125 | 2026.08.17 23:52:15.427814 [ 432023 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000001 - log-0000000001 | ||
| 1126 | 2026.08.17 23:52:15.427867 [ 432091 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1127 | 2026.08.17 23:52:15.427879 [ 432091 ] {} <Information> oximeter.version (ReplicatedMergeTreeAttachThread): Table is initialized | ||
| 1128 | 2026.08.17 23:52:15.427959 [ 432063 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 1129 | 2026.08.17 23:52:15.433938 [ 432023 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000001 to queue with type GET_PART with virtual parts [all_1_1_0] | ||
| 1130 | 2026.08.17 23:52:15.433945 [ 432023 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 1131 | 2026.08.17 23:52:15.437340 [ 432057 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1132 | 2026.08.17 23:52:15.437357 [ 432058 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000001 to queue with type GET_PART with virtual parts [all_1_1_0] | ||
| 1133 | 2026.08.17 23:52:15.437382 [ 432058 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 1134 | 2026.08.17 23:52:15.437440 [ 432076 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72966033408, merge_with_ttl_allowedtrue) | ||
| 1135 | 2026.08.17 23:52:15.437466 [ 432023 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1136 | 2026.08.17 23:52:15.437485 [ 432076 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 1137 | 2026.08.17 23:52:15.437506 [ 432023 ] {} <Information> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeAttachThread): Table is initialized | ||
| 1138 | 2026.08.17 23:52:15.437454 [ 432099 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000001 - log-0000000001 | ||
| 1139 | 2026.08.17 23:52:15.437514 [ 432107 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72966033408, merge_with_ttl_allowedtrue) | ||
| 1140 | 2026.08.17 23:52:15.437529 [ 432107 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 1141 | 2026.08.17 23:52:15.437546 [ 432057 ] {} <Information> oximeter.timeseries_schema_local (ReplicatedMergeTreeAttachThread): Table is initialized | ||
| 1142 | 2026.08.17 23:52:15.437724 [ 432058 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1143 | 2026.08.17 23:52:15.437744 [ 432058 ] {} <Information> oximeter.fields_uuid_local (ReplicatedMergeTreeAttachThread): Table is initialized | ||
| 1144 | 2026.08.17 23:52:15.437823 [ 432061 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeCleanupThread): Checking 1 blocks (1 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/timeseries_schema_local/blocks | ||
| 1145 | 2026.08.17 23:52:15.437826 [ 432056 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72966031360, merge_with_ttl_allowedtrue) | ||
| 1146 | 2026.08.17 23:52:15.437825 [ 432047 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeCleanupThread): Checking 2 blocks (2 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/measurements_cumulativef64_local/blocks | ||
| 1147 | 2026.08.17 23:52:15.437840 [ 432056 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 1148 | 2026.08.17 23:52:15.440422 [ 432099 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000001 to queue with type GET_PART with virtual parts [all_1_1_0] | ||
| 1149 | 2026.08.17 23:52:15.440428 [ 432099 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 1150 | 2026.08.17 23:52:15.440558 [ 432058 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeCleanupThread): Checking 2 blocks (2 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/fields_uuid_local/blocks | ||
| 1151 | 2026.08.17 23:52:15.440569 [ 432209 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Fetching part all_1_1_0 from default:/clickhouse/tables/1/measurements_cumulativef64_local/replicas/3 | ||
| 1152 | 2026.08.17 23:52:15.440640 [ 432099 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1153 | 2026.08.17 23:52:15.440651 [ 432099 ] {} <Information> oximeter.fields_i64_local (ReplicatedMergeTreeAttachThread): Table is initialized | ||
| 1154 | 2026.08.17 23:52:15.440686 [ 432219 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Fetching part all_1_1_0 from default:/clickhouse/tables/1/fields_uuid_local/replicas/3 | ||
| 1155 | 2026.08.17 23:52:15.440706 [ 432209 ] {} <Trace> ReadWriteBufferFromHTTP: Sending request to http://[::1]:19403/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Fmeasurements_cumulativef64_local%2Freplicas%2F3&part=all_1_1_0&client_protocol_version=8&compress=false | ||
| 1156 | 2026.08.17 23:52:15.440778 [ 432209 ] {} <Trace> HTTPSessionAdapter: Created HTTP(S) session with ::1:19403 (::1:19403) | ||
| 1157 | 2026.08.17 23:52:15.440799 [ 432219 ] {} <Trace> ReadWriteBufferFromHTTP: Sending request to http://[::1]:19403/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_uuid_local%2Freplicas%2F3&part=all_1_1_0&client_protocol_version=8&compress=false | ||
| 1158 | 2026.08.17 23:52:15.440801 [ 432019 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72965963776, merge_with_ttl_allowedtrue) | ||
| 1159 | 2026.08.17 23:52:15.440818 [ 432019 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 1160 | 2026.08.17 23:52:15.440832 [ 432219 ] {} <Trace> HTTPSessionAdapter: Created HTTP(S) session with ::1:19403 (::1:19403) | ||
| 1161 | 2026.08.17 23:52:15.440958 [ 431908 ] {} <Information> UserDefinedSQLObjectsLoaderFromDisk: Loading user defined objects from /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/data/user_defined/ | ||
| 1162 | 2026.08.17 23:52:15.440980 [ 431908 ] {} <Debug> UserDefinedSQLObjectsLoaderFromDisk: User defined objects loaded | ||
| 1163 | 2026.08.17 23:52:15.440982 [ 431908 ] {} <Debug> Application: Loaded metadata. | ||
| 1164 | 2026.08.17 23:52:15.441003 [ 431908 ] {} <Information> Application: Tasks stats provider: procfs | ||
| 1165 | 2026.08.17 23:52:15.441005 [ 431908 ] {} <Information> Application: It looks like the process has no CAP_SYS_NICE capability, the setting 'os_thread_priority' will have no effect. It could happen due to incorrect ClickHouse package installation. You could resolve the problem manually with 'sudo setcap cap_sys_nice=+ep /work/oxidecomputer/omicron/out/clickhouse/clickhouse'. Note that it will not work on 'nosuid' mounted filesystems. | ||
| 1166 | 2026.08.17 23:52:15.441039 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Fetching part all_1_1_0 from default:/clickhouse/tables/1/fields_i64_local/replicas/3 | ||
| 1167 | 2026.08.17 23:52:15.441094 [ 431908 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. | ||
| 1168 | 2026.08.17 23:52:15.441134 [ 432220 ] {} <Trace> ReadWriteBufferFromHTTP: Sending request to http://[::1]:19403/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_i64_local%2Freplicas%2F3&part=all_1_1_0&client_protocol_version=8&compress=false | ||
| 1169 | 2026.08.17 23:52:15.441187 [ 432220 ] {} <Trace> HTTPSessionAdapter: Created HTTP(S) session with ::1:19403 (::1:19403) | ||
| 1170 | 2026.08.17 23:52:15.441566 [ 431908 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 13.38 MiB, peak 13.38 MiB, free memory in arenas 22.94 MiB, will set to 390.07 MiB (RSS), difference: 376.69 MiB | ||
| 1171 | 2026.08.17 23:52:15.441854 [ 432089 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeCleanupThread): Checking 2 blocks (2 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/fields_i64_local/blocks | ||
| 1172 | 2026.08.17 23:52:15.442003 [ 431908 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 7.277e-05 sec. | ||
| 1173 | 2026.08.17 23:52:15.442104 [ 432273 ] {} <Debug> DDLWorker: Starting DDLWorker thread | ||
| 1174 | 2026.08.17 23:52:15.442104 [ 432268 ] {} <Debug> DDLWorker: Started DDLWorker cleanup thread | ||
| 1175 | 2026.08.17 23:52:15.442111 [ 432273 ] {} <Debug> DDLWorker: Initializing DDLWorker thread | ||
| 1176 | 2026.08.17 23:52:15.442125 [ 431908 ] {} <Information> Application: Listening for http://[::1]:19301 | ||
| 1177 | 2026.08.17 23:52:15.442145 [ 431908 ] {} <Information> Application: Listening for native protocol (tcp): [::1]:19201 | ||
| 1178 | 2026.08.17 23:52:15.442152 [ 431908 ] {} <Information> Application: Ready for connections. | ||
| 1179 | 2026.08.17 23:52:15.443160 [ 432209 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1180 | 2026.08.17 23:52:15.443178 [ 432209 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.91 GiB. | ||
| 1181 | 2026.08.17 23:52:15.443184 [ 432209 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Fetcher): Disk for fetch is not provided, getting disk from reservation default with type 'local' | ||
| 1182 | 2026.08.17 23:52:15.443187 [ 432219 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1183 | 2026.08.17 23:52:15.443194 [ 432209 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Fetcher): Downloading part all_1_1_0 onto disk default. | ||
| 1184 | 2026.08.17 23:52:15.443200 [ 432219 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.91 GiB. | ||
| 1185 | 2026.08.17 23:52:15.443188 [ 432220 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1186 | 2026.08.17 23:52:15.443209 [ 432219 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Disk for fetch is not provided, getting disk from reservation default with type 'local' | ||
| 1187 | 2026.08.17 23:52:15.443217 [ 432219 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Downloading part all_1_1_0 onto disk default. | ||
| 1188 | 2026.08.17 23:52:15.443219 [ 432220 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.91 GiB. | ||
| 1189 | 2026.08.17 23:52:15.443229 [ 432220 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Fetcher): Disk for fetch is not provided, getting disk from reservation default with type 'local' | ||
| 1190 | 2026.08.17 23:52:15.443235 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Fetcher): Downloading part all_1_1_0 onto disk default. | ||
| 1191 | 2026.08.17 23:52:15.443716 [ 432219 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Download of part all_1_1_0 onto disk default finished. | ||
| 1192 | 2026.08.17 23:52:15.443716 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Fetcher): Download of part all_1_1_0 onto disk default finished. | ||
| 1193 | 2026.08.17 23:52:15.443729 [ 432219 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Renaming temporary part tmp-fetch_all_1_1_0 to all_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1194 | 2026.08.17 23:52:15.443730 [ 432220 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Renaming temporary part tmp-fetch_all_1_1_0 to all_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1195 | 2026.08.17 23:52:15.443761 [ 432219 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Committing part all_1_1_0 to zookeeper | ||
| 1196 | 2026.08.17 23:52:15.443763 [ 432209 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Fetcher): Download of part all_1_1_0 onto disk default finished. | ||
| 1197 | 2026.08.17 23:52:15.443761 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Committing part all_1_1_0 to zookeeper | ||
| 1198 | 2026.08.17 23:52:15.443780 [ 432209 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Renaming temporary part tmp-fetch_all_1_1_0 to all_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1199 | 2026.08.17 23:52:15.445736 [ 432209 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Committing part all_1_1_0 to zookeeper | ||
| 1200 | 2026.08.17 23:52:15.450613 [ 431938 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56620 | ||
| 1201 | 2026.08.17 23:52:15.450815 [ 431938 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 1202 | 2026.08.17 23:52:15.450832 [ 431938 ] {} <Debug> TCP-Session: f8dc188c-ca4e-4008-be73-58346a54e0e4 Authenticating user 'default' from [::1]:56620 | ||
| 1203 | 2026.08.17 23:52:15.450843 [ 431938 ] {} <Debug> TCP-Session: f8dc188c-ca4e-4008-be73-58346a54e0e4 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 1204 | 2026.08.17 23:52:15.450846 [ 431938 ] {} <Debug> TCP-Session: f8dc188c-ca4e-4008-be73-58346a54e0e4 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 1205 | 2026.08.17 23:52:15.450916 [ 431938 ] {} <Trace> ContextAccess (default): Settings: readonly = 0, allow_ddl = true, allow_introspection_functions = false | ||
| 1206 | 2026.08.17 23:52:15.450927 [ 431938 ] {} <Trace> ContextAccess (default): List of all grants: GRANT SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* | ||
| 1207 | 2026.08.17 23:52:15.450930 [ 431938 ] {} <Trace> ContextAccess (default): List of all grants including implicit: GRANT SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* | ||
| 1208 | 2026.08.17 23:52:15.453040 [ 432273 ] {} <Debug> DDLWorker: Initialized DDLWorker thread | ||
| 1209 | 2026.08.17 23:52:15.453047 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 1210 | 2026.08.17 23:52:15.453050 [ 432273 ] {} <Trace> DDLWorker: Don't have unfinished tasks after restarting | ||
| 1211 | 2026.08.17 23:52:15.453052 [ 432268 ] {} <Debug> DDLWorker: Cleaning queue | ||
| 1212 | 2026.08.17 23:52:15.453154 [ 432219 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Part all_1_1_0 committed to zookeeper | ||
| 1213 | 2026.08.17 23:52:15.453178 [ 432219 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Fetched part all_1_1_0 from default:/clickhouse/tables/1/fields_uuid_local/replicas/3 | ||
| 1214 | 2026.08.17 23:52:15.453228 [ 432033 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72965681152, merge_with_ttl_allowedtrue) | ||
| 1215 | 2026.08.17 23:52:15.453238 [ 432033 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 1216 | 2026.08.17 23:52:15.454851 [ 431939 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56622 | ||
| 1217 | 2026.08.17 23:52:15.455416 [ 431939 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 1218 | 2026.08.17 23:52:15.455433 [ 431939 ] {} <Debug> TCP-Session: 73eaf47d-a602-48f9-b745-357cd4ccad2a Authenticating user 'default' from [::1]:56622 | ||
| 1219 | 2026.08.17 23:52:15.455442 [ 431939 ] {} <Debug> TCP-Session: 73eaf47d-a602-48f9-b745-357cd4ccad2a Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 1220 | 2026.08.17 23:52:15.455445 [ 431939 ] {} <Debug> TCP-Session: 73eaf47d-a602-48f9-b745-357cd4ccad2a Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 1221 | 2026.08.17 23:52:15.458249 [ 432209 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Part all_1_1_0 committed to zookeeper | ||
| 1222 | 2026.08.17 23:52:15.458278 [ 432209 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Fetched part all_1_1_0 from default:/clickhouse/tables/1/measurements_cumulativef64_local/replicas/3 | ||
| 1223 | 2026.08.17 23:52:15.458331 [ 432026 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72965597184, merge_with_ttl_allowedtrue) | ||
| 1224 | 2026.08.17 23:52:15.458346 [ 432026 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 1225 | 2026.08.17 23:52:15.461989 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Part all_1_1_0 committed to zookeeper | ||
| 1226 | 2026.08.17 23:52:15.462020 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Fetched part all_1_1_0 from default:/clickhouse/tables/1/fields_i64_local/replicas/3 | ||
| 1227 | 2026.08.17 23:52:15.462048 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=20, queue_size=20, entries=query-0000000000..query-0000000019, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=none | ||
| 1228 | 2026.08.17 23:52:15.462065 [ 432273 ] {} <Debug> DDLWorker: Will schedule 20 tasks starting from query-0000000000 | ||
| 1229 | 2026.08.17 23:52:15.462068 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000000 | ||
| 1230 | 2026.08.17 23:52:15.462087 [ 432060 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72965539840, merge_with_ttl_allowedtrue) | ||
| 1231 | 2026.08.17 23:52:15.462102 [ 432060 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 1232 | 2026.08.17 23:52:15.466108 [ 431938 ] {} <Trace> TCP-Session: f8dc188c-ca4e-4008-be73-58346a54e0e4 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 1233 | 2026.08.17 23:52:15.466238 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> executeQuery: Query span trace_id for opentelemetry log: 17cd4ea7-f7a3-0c99-30b1-152c08e0f4fe | ||
| 1234 | 2026.08.17 23:52:15.466319 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Debug> executeQuery: (from [::1]:56620) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 1235 | 2026.08.17 23:52:15.466651 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 1236 | 2026.08.17 23:52:15.466791 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 1237 | 2026.08.17 23:52:15.466909 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 1238 | 2026.08.17 23:52:15.466918 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name IN ('virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 1239 | 2026.08.17 23:52:15.466977 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 1240 | 2026.08.17 23:52:15.467009 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1241 | 2026.08.17 23:52:15.467044 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1242 | 2026.08.17 23:52:15.467149 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Key condition: (column 0 in 1-element set) | ||
| 1243 | 2026.08.17 23:52:15.467162 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 1244 | 2026.08.17 23:52:15.467176 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1245 | 2026.08.17 23:52:15.467185 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1246 | 2026.08.17 23:52:15.467188 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found continuous range in 2 steps | ||
| 1247 | 2026.08.17 23:52:15.467200 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 1248 | 2026.08.17 23:52:15.467205 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 1249 | 2026.08.17 23:52:15.467226 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "in(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1250 | 2026.08.17 23:52:15.467232 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 1 rows starting from 0 | ||
| 1251 | 2026.08.17 23:52:15.467881 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.001628 sec., 614.2506142506143 rows/sec., 75.58 KiB/sec. | ||
| 1252 | 2026.08.17 23:52:15.467921 [ 431938 ] {0d89a1ac-fe45-44e8-bcc3-cf4f1cc18c70} <Debug> TCPHandler: Processed in 0.001983452 sec. | ||
| 1253 | 2026.08.17 23:52:15.468468 [ 431938 ] {} <Trace> TCP-Session: f8dc188c-ca4e-4008-be73-58346a54e0e4 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 1254 | 2026.08.17 23:52:15.469665 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> executeQuery: Query span trace_id for opentelemetry log: d895175b-45ed-face-bc9d-d84298a27e95 | ||
| 1255 | 2026.08.17 23:52:15.469722 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Debug> executeQuery: (from [::1]:56620) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 1256 | 2026.08.17 23:52:15.469873 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 1257 | 2026.08.17 23:52:15.469963 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 1258 | 2026.08.17 23:52:15.470058 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 1259 | 2026.08.17 23:52:15.470065 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name IN ('virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 1260 | 2026.08.17 23:52:15.470118 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 1261 | 2026.08.17 23:52:15.470140 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1262 | 2026.08.17 23:52:15.470172 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1263 | 2026.08.17 23:52:15.470232 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Key condition: (column 0 in 1-element set) | ||
| 1264 | 2026.08.17 23:52:15.470239 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 1265 | 2026.08.17 23:52:15.470250 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1266 | 2026.08.17 23:52:15.470254 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1267 | 2026.08.17 23:52:15.470258 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found continuous range in 2 steps | ||
| 1268 | 2026.08.17 23:52:15.470266 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 1269 | 2026.08.17 23:52:15.470270 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 1270 | 2026.08.17 23:52:15.470282 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "in(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1271 | 2026.08.17 23:52:15.470287 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 1 rows starting from 0 | ||
| 1272 | 2026.08.17 23:52:15.470782 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.001098 sec., 910.7468123861566 rows/sec., 112.06 KiB/sec. | ||
| 1273 | 2026.08.17 23:52:15.471885 [ 431938 ] {e67635de-bfa3-4c43-951e-dfdf7580ec66} <Debug> TCPHandler: Processed in 0.003474152 sec. | ||
| 1274 | 2026.08.17 23:52:15.472204 [ 431939 ] {} <Trace> TCP-Session: 73eaf47d-a602-48f9-b745-357cd4ccad2a Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 1275 | 2026.08.17 23:52:15.472272 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> executeQuery: Query span trace_id for opentelemetry log: 8831ab74-53b2-df43-6759-c1eaeb9ec4da | ||
| 1276 | 2026.08.17 23:52:15.472398 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> executeQuery: (from [::1]:56622) SELECT fields_i64_pivot.cpu_id AS cpu_id, fields_uuid_pivot.instance_id AS instance_id, fields_uuid_pivot.project_id AS project_id, fields_uuid_pivot.timeseries_key AS timeseries_key FROM ( SELECT timeseries_key, anyIf(field_value, field_name = 'instance_id') AS instance_id, anyIf(field_value, field_name = 'project_id') AS project_id FROM oximeter.fields_uuid WHERE timeseries_name = 'virtual_machine:cpu_busy' GROUP BY timeseries_key ) AS fields_uuid_pivot JOIN ( SELECT timeseries_key, anyIf(field_value, field_name = 'cpu_id') AS cpu_id FROM oximeter.fields_i64 WHERE timeseries_name = 'virtual_machine:cpu_busy' GROUP BY timeseries_key ) AS fields_i64_pivot ON fields_uuid_pivot.timeseries_key = fields_i64_pivot.timeseries_key (stage: Complete) | ||
| 1277 | 2026.08.17 23:52:15.472686 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 1278 | 2026.08.17 23:52:15.472807 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 1279 | 2026.08.17 23:52:15.472931 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 1280 | 2026.08.17 23:52:15.474695 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 1281 | 2026.08.17 23:52:15.474803 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 1282 | 2026.08.17 23:52:15.474922 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 1283 | 2026.08.17 23:52:15.474929 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name = 'virtual_machine:cpu_busy'" moved to PREWHERE | ||
| 1284 | 2026.08.17 23:52:15.474987 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64_local | ||
| 1285 | 2026.08.17 23:52:15.475003 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1286 | 2026.08.17 23:52:15.475030 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1287 | 2026.08.17 23:52:15.475062 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> HashJoin: (0x7feaf7343518) Datatype: EMPTY, kind: Inner, strictness: All, right header: fields_i64_pivot.timeseries_key UInt64 UInt64(size = 0), cpu_id Int64 Int64(size = 0) | ||
| 1288 | 2026.08.17 23:52:15.475067 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> HashJoin: (0x7feaf7343518) Keys: [(timeseries_key) = (fields_i64_pivot.timeseries_key)] | ||
| 1289 | 2026.08.17 23:52:15.475257 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 1290 | 2026.08.17 23:52:15.476523 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 1291 | 2026.08.17 23:52:15.476649 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 1292 | 2026.08.17 23:52:15.476656 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name = 'virtual_machine:cpu_busy'" moved to PREWHERE | ||
| 1293 | 2026.08.17 23:52:15.476724 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid_local | ||
| 1294 | 2026.08.17 23:52:15.476740 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1295 | 2026.08.17 23:52:15.476778 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1296 | 2026.08.17 23:52:15.476780 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1297 | 2026.08.17 23:52:15.476791 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> JoiningTransform: Before join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0)' | ||
| 1298 | 2026.08.17 23:52:15.476806 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> JoiningTransform: After join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0), cpu_id Int64 Int64(size = 0)' | ||
| 1299 | 2026.08.17 23:52:15.477078 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Key condition: (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']) | ||
| 1300 | 2026.08.17 23:52:15.477148 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_1_1_0 (2 marks) | ||
| 1301 | 2026.08.17 23:52:15.477168 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1302 | 2026.08.17 23:52:15.477176 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1303 | 2026.08.17 23:52:15.477181 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 1304 | 2026.08.17 23:52:15.477242 [ 432264 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 1305 | 2026.08.17 23:52:15.477248 [ 432264 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1306 | 2026.08.17 23:52:15.477250 [ 432264 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1307 | 2026.08.17 23:52:15.477253 [ 432264 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 1308 | 2026.08.17 23:52:15.477301 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Selected 2/2 parts by partition key, 2 parts by primary key, 2/2 marks by primary key, 2 marks to read from 2 ranges | ||
| 1309 | 2026.08.17 23:52:15.477312 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 1310 | 2026.08.17 23:52:15.477358 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> MergeTreeReadPool: min_marks_for_concurrent_read=24 | ||
| 1311 | 2026.08.17 23:52:15.477366 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Reading approx. 32 rows with 2 streams | ||
| 1312 | 2026.08.17 23:52:15.477379 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1313 | 2026.08.17 23:52:15.477388 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1314 | 2026.08.17 23:52:15.477534 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Key condition: (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']) | ||
| 1315 | 2026.08.17 23:52:15.478830 [ 432257 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 1316 | 2026.08.17 23:52:15.478849 [ 432257 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1317 | 2026.08.17 23:52:15.478854 [ 432257 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1318 | 2026.08.17 23:52:15.478857 [ 432257 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 1319 | 2026.08.17 23:52:15.478907 [ 432257 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_1_1_0 (2 marks) | ||
| 1320 | 2026.08.17 23:52:15.478910 [ 432257 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1321 | 2026.08.17 23:52:15.478916 [ 432257 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1322 | 2026.08.17 23:52:15.478921 [ 432257 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 1323 | 2026.08.17 23:52:15.478983 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Selected 2/2 parts by partition key, 2 parts by primary key, 2/2 marks by primary key, 2 marks to read from 2 ranges | ||
| 1324 | 2026.08.17 23:52:15.478995 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 1325 | 2026.08.17 23:52:15.479034 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> MergeTreeReadPool: min_marks_for_concurrent_read=24 | ||
| 1326 | 2026.08.17 23:52:15.479041 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Reading approx. 16 rows with 2 streams | ||
| 1327 | 2026.08.17 23:52:15.479056 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1328 | 2026.08.17 23:52:15.479068 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1329 | 2026.08.17 23:52:15.479200 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> JoiningTransform: Before join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0)' | ||
| 1330 | 2026.08.17 23:52:15.479216 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> JoiningTransform: After join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0), cpu_id Int64 Int64(size = 0)' | ||
| 1331 | 2026.08.17 23:52:15.479985 [ 432259 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> AggregatingTransform: Aggregating | ||
| 1332 | 2026.08.17 23:52:15.480001 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000000: Task has been already processed | ||
| 1333 | 2026.08.17 23:52:15.480683 [ 432259 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> Aggregator: Aggregation method: key64 | ||
| 1334 | 2026.08.17 23:52:15.480689 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000001 | ||
| 1335 | 2026.08.17 23:52:15.480045 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> AggregatingTransform: Aggregating | ||
| 1336 | 2026.08.17 23:52:15.480712 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> Aggregator: Aggregation method: key64 | ||
| 1337 | 2026.08.17 23:52:15.480731 [ 432259 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.001599851 sec. (5000.466 rows/sec., 83.02 KiB/sec.) | ||
| 1338 | 2026.08.17 23:52:15.480732 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.001599321 sec. (5002.123 rows/sec., 83.04 KiB/sec.) | ||
| 1339 | 2026.08.17 23:52:15.480816 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> Aggregator: Merging aggregated data | ||
| 1340 | 2026.08.17 23:52:15.480821 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> Aggregator: Statistics updated for key=1559201803930241434: new sum_of_sizes=16, median_size=8 | ||
| 1341 | 2026.08.17 23:52:15.480928 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000001: Task has been already processed | ||
| 1342 | 2026.08.17 23:52:15.480933 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000002 | ||
| 1343 | 2026.08.17 23:52:15.481205 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000002: Task has been already processed | ||
| 1344 | 2026.08.17 23:52:15.481208 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000003 | ||
| 1345 | 2026.08.17 23:52:15.481326 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000003: Task has been already processed | ||
| 1346 | 2026.08.17 23:52:15.481333 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000004 | ||
| 1347 | 2026.08.17 23:52:15.481444 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000004: Task has been already processed | ||
| 1348 | 2026.08.17 23:52:15.481446 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000005 | ||
| 1349 | 2026.08.17 23:52:15.481631 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000005: Task has been already processed | ||
| 1350 | 2026.08.17 23:52:15.483010 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000006 | ||
| 1351 | 2026.08.17 23:52:15.483205 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000006: Task has been already processed | ||
| 1352 | 2026.08.17 23:52:15.483213 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000007 | ||
| 1353 | 2026.08.17 23:52:15.483332 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000007: Task has been already processed | ||
| 1354 | 2026.08.17 23:52:15.483334 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000008 | ||
| 1355 | 2026.08.17 23:52:15.483595 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000008: Task has been already processed | ||
| 1356 | 2026.08.17 23:52:15.483598 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000009 | ||
| 1357 | 2026.08.17 23:52:15.483830 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000009: Task has been already processed | ||
| 1358 | 2026.08.17 23:52:15.483835 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000010 | ||
| 1359 | 2026.08.17 23:52:15.483967 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000010: Task has been already processed | ||
| 1360 | 2026.08.17 23:52:15.483972 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000011 | ||
| 1361 | 2026.08.17 23:52:15.484115 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000011: Task has been already processed | ||
| 1362 | 2026.08.17 23:52:15.484118 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000012 | ||
| 1363 | 2026.08.17 23:52:15.484292 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000012: Task has been already processed | ||
| 1364 | 2026.08.17 23:52:15.484297 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000013 | ||
| 1365 | 2026.08.17 23:52:15.484458 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000013: Task has been already processed | ||
| 1366 | 2026.08.17 23:52:15.484461 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000014 | ||
| 1367 | 2026.08.17 23:52:15.484589 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000014: Task has been already processed | ||
| 1368 | 2026.08.17 23:52:15.484591 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000015 | ||
| 1369 | 2026.08.17 23:52:15.484697 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000015: Task has been already processed | ||
| 1370 | 2026.08.17 23:52:15.484699 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000016 | ||
| 1371 | 2026.08.17 23:52:15.484802 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000016: Task has been already processed | ||
| 1372 | 2026.08.17 23:52:15.486185 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000017 | ||
| 1373 | 2026.08.17 23:52:15.486327 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000017: Task has been already processed | ||
| 1374 | 2026.08.17 23:52:15.486331 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000018 | ||
| 1375 | 2026.08.17 23:52:15.486532 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000018: Task has been already processed | ||
| 1376 | 2026.08.17 23:52:15.486535 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000019 | ||
| 1377 | 2026.08.17 23:52:15.486734 [ 432273 ] {} <Debug> DDLWorker: Will not execute task query-0000000019: Task has been already processed | ||
| 1378 | 2026.08.17 23:52:15.486738 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 1379 | 2026.08.17 23:52:15.511345 [ 432259 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> AggregatingTransform: Aggregating | ||
| 1380 | 2026.08.17 23:52:15.514014 [ 432259 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> Aggregator: Aggregation method: key64 | ||
| 1381 | 2026.08.17 23:52:15.514044 [ 432259 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.036585987 sec. (437.326 rows/sec., 11.10 KiB/sec.) | ||
| 1382 | 2026.08.17 23:52:15.514141 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> AggregatingTransform: Aggregating | ||
| 1383 | 2026.08.17 23:52:15.514150 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> Aggregator: Aggregation method: key64 | ||
| 1384 | 2026.08.17 23:52:15.514166 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.036725807 sec. (435.661 rows/sec., 11.06 KiB/sec.) | ||
| 1385 | 2026.08.17 23:52:15.514168 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> Aggregator: Merging aggregated data | ||
| 1386 | 2026.08.17 23:52:15.514172 [ 432266 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> Aggregator: Statistics updated for key=11757313445726935263: new sum_of_sizes=16, median_size=8 | ||
| 1387 | 2026.08.17 23:52:15.514812 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> executeQuery: Read 48 rows, 3.39 KiB in 0.042521 sec., 1128.8539780343829 rows/sec., 79.74 KiB/sec. | ||
| 1388 | 2026.08.17 23:52:15.514960 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Trace> HashJoin: (0x7feaf7343518) Join data is being destroyed, 12800 bytes and 16 rows in hash table | ||
| 1389 | 2026.08.17 23:52:15.515033 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> MemoryTracker: Peak memory usage (for query): 237.28 KiB. | ||
| 1390 | 2026.08.17 23:52:15.515036 [ 431939 ] {2cd3096b-0b94-41cc-9341-f774a27f6f5e} <Debug> TCPHandler: Processed in 0.04289307 sec. | ||
| 1391 | 2026.08.17 23:52:15.515451 [ 431939 ] {} <Trace> TCP-Session: 73eaf47d-a602-48f9-b745-357cd4ccad2a Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 1392 | 2026.08.17 23:52:15.516553 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> executeQuery: Query span trace_id for opentelemetry log: c7fdb492-37c4-4a63-0e77-f1138242d7d3 | ||
| 1393 | 2026.08.17 23:52:15.516659 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Debug> executeQuery: (from [::1]:56622) SELECT timeseries_key, start_time, timestamp, datum FROM oximeter.measurements_cumulativef64 WHERE timeseries_name = 'virtual_machine:cpu_busy' AND ((greater(timestamp, '2000-01-01 00:00:00.000000000') AND timeseries_key IN (462795175800602962,2318161133194800099,3810615656589673070,5210923722195156639,5419464027902213652,6772052453391040098,7613129139487427710,11185254949005441257,11365707485692726787,11873430000991232922,12428098974121224233,13097606626804240593,15203798703989247109,15279762139926950523,15520859351690753739,15994013491234619255))) ORDER BY timeseries_key, start_time, timestamp LIMIT 1000001 (stage: Complete) | ||
| 1394 | 2026.08.17 23:52:15.517003 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 1395 | 2026.08.17 23:52:15.517210 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 1396 | 2026.08.17 23:52:15.517386 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 1 | ||
| 1397 | 2026.08.17 23:52:15.517399 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "(timestamp > '2000-01-01 00:00:00.000000000') AND (timeseries_key IN (462795175800602962, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6772052453391040098, 7613129139487427710, 11185254949005441257, 11365707485692726787, 11873430000991232922, 12428098974121224233, 13097606626804240593, 15203798703989247109, 15279762139926950523, 15520859351690753739, 15994013491234619255)) AND (timeseries_name = 'virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 1398 | 2026.08.17 23:52:15.517527 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64_local | ||
| 1399 | 2026.08.17 23:52:15.517564 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1400 | 2026.08.17 23:52:15.517599 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1401 | 2026.08.17 23:52:15.517718 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Key condition: (column 3 in ('946684800', +Inf)), (column 1 in 16-element set), and, (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']), and | ||
| 1402 | 2026.08.17 23:52:15.517785 [ 432259 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Used generic exclusion search over index for part all_1_1_0 with 1 steps | ||
| 1403 | 2026.08.17 23:52:15.517800 [ 432265 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Used generic exclusion search over index for part all_0_0_0 with 1 steps | ||
| 1404 | 2026.08.17 23:52:15.517892 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Selected 2/2 parts by partition key, 2 parts by primary key, 2/2 marks by primary key, 2 marks to read from 2 ranges | ||
| 1405 | 2026.08.17 23:52:15.517902 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Spreading ranges among streams with order | ||
| 1406 | 2026.08.17 23:52:15.517973 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 3 steps: "greater(timestamp, '2000-01-01 00:00:00.000000000')", "in(timeseries_key, (462795175800602962, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6772052453391040098, 7613129139487427710, 11185254949005441257, 11365707485692726787, 11873430000991232922, 12428098974121224233, 13097606626804240593, 15203798703989247109, 15279762139926950523, 15520859351690753739, 15994013491234619255))", "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1407 | 2026.08.17 23:52:15.517978 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_1_1_0, approx. 400 rows starting from 0 | ||
| 1408 | 2026.08.17 23:52:15.518014 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 3 steps: "greater(timestamp, '2000-01-01 00:00:00.000000000')", "in(timeseries_key, (462795175800602962, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6772052453391040098, 7613129139487427710, 11185254949005441257, 11365707485692726787, 11873430000991232922, 12428098974121224233, 13097606626804240593, 15203798703989247109, 15279762139926950523, 15520859351690753739, 15994013491234619255))", "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1409 | 2026.08.17 23:52:15.518018 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 400 rows starting from 0 | ||
| 1410 | 2026.08.17 23:52:15.518597 [ 432293 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Debug> MergingSortedTransform: Merge sorted 1 blocks, 800 rows in 0 sec. | ||
| 1411 | 2026.08.17 23:52:15.519838 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Debug> executeQuery: Read 800 rows, 51.56 KiB in 0.003272 sec., 244498.77750611247 rows/sec., 15.39 MiB/sec. | ||
| 1412 | 2026.08.17 23:52:15.519919 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Debug> MemoryTracker: Peak memory usage (for query): 253.84 KiB. | ||
| 1413 | 2026.08.17 23:52:15.519922 [ 431939 ] {37c1ce0b-c374-4eb2-b695-1bd0f908eaea} <Debug> TCPHandler: Processed in 0.004595443 sec. | ||
| 1414 | 2026.08.17 23:52:15.523502 [ 431938 ] {} <Trace> TCP-Session: f8dc188c-ca4e-4008-be73-58346a54e0e4 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 1415 | 2026.08.17 23:52:15.523572 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> executeQuery: Query span trace_id for opentelemetry log: 15eeef4d-3fba-0978-15f3-3750996c230c | ||
| 1416 | 2026.08.17 23:52:15.523628 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Debug> executeQuery: (from [::1]:56620) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 1417 | 2026.08.17 23:52:15.523812 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 1418 | 2026.08.17 23:52:15.523931 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 1419 | 2026.08.17 23:52:15.524043 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 1420 | 2026.08.17 23:52:15.524050 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name IN ('virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 1421 | 2026.08.17 23:52:15.524108 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 1422 | 2026.08.17 23:52:15.524133 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1423 | 2026.08.17 23:52:15.524164 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1424 | 2026.08.17 23:52:15.524228 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Key condition: (column 0 in 1-element set) | ||
| 1425 | 2026.08.17 23:52:15.524237 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 1426 | 2026.08.17 23:52:15.524245 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1427 | 2026.08.17 23:52:15.524249 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1428 | 2026.08.17 23:52:15.524252 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found continuous range in 2 steps | ||
| 1429 | 2026.08.17 23:52:15.524261 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 1430 | 2026.08.17 23:52:15.524265 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 1431 | 2026.08.17 23:52:15.524292 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "in(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1432 | 2026.08.17 23:52:15.524297 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 1 rows starting from 0 | ||
| 1433 | 2026.08.17 23:52:15.524784 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.001201 sec., 832.6394671107411 rows/sec., 102.45 KiB/sec. | ||
| 1434 | 2026.08.17 23:52:15.525973 [ 431938 ] {38470794-631e-4b66-9b74-f35add94d5a8} <Debug> TCPHandler: Processed in 0.002523882 sec. | ||
| 1435 | 2026.08.17 23:52:15.526412 [ 431938 ] {} <Trace> TCP-Session: f8dc188c-ca4e-4008-be73-58346a54e0e4 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 1436 | 2026.08.17 23:52:15.526460 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> executeQuery: Query span trace_id for opentelemetry log: 74caedc2-afa0-8dea-ef2a-f7cb29050a2b | ||
| 1437 | 2026.08.17 23:52:15.526508 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Debug> executeQuery: (from [::1]:56620) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 1438 | 2026.08.17 23:52:15.526652 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 1439 | 2026.08.17 23:52:15.526757 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 1440 | 2026.08.17 23:52:15.526845 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 1441 | 2026.08.17 23:52:15.526852 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name IN ('virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 1442 | 2026.08.17 23:52:15.526904 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 1443 | 2026.08.17 23:52:15.526925 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1444 | 2026.08.17 23:52:15.526961 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1445 | 2026.08.17 23:52:15.527022 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Key condition: (column 0 in 1-element set) | ||
| 1446 | 2026.08.17 23:52:15.527029 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 1447 | 2026.08.17 23:52:15.527037 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1448 | 2026.08.17 23:52:15.527041 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1449 | 2026.08.17 23:52:15.527044 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found continuous range in 2 steps | ||
| 1450 | 2026.08.17 23:52:15.527051 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 1451 | 2026.08.17 23:52:15.527055 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 1452 | 2026.08.17 23:52:15.527068 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "in(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1453 | 2026.08.17 23:52:15.527072 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 1 rows starting from 0 | ||
| 1454 | 2026.08.17 23:52:15.527499 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.00103 sec., 970.8737864077669 rows/sec., 119.46 KiB/sec. | ||
| 1455 | 2026.08.17 23:52:15.527531 [ 431938 ] {e2d43755-7886-470f-841e-c2d0121e7470} <Debug> TCPHandler: Processed in 0.001164021 sec. | ||
| 1456 | 2026.08.17 23:52:15.527938 [ 431939 ] {} <Trace> TCP-Session: 73eaf47d-a602-48f9-b745-357cd4ccad2a Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 1457 | 2026.08.17 23:52:15.527992 [ 432271 ] {} <Error> ZooKeeperClient: Code: 33. DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4. (CANNOT_READ_ALL_DATA), Stack trace (when copying this message, always include the lines below): | ||
| 1458 | |||
| 1459 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1460 | 1. DB::Exception::Exception<unsigned long&, String>(int, FormatStringHelperImpl<std::type_identity<unsigned long&>::type, std::type_identity<String>::type>, unsigned long&, String&&) @ 0x0000000007354cb4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1461 | 2. DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x0000000007354bb7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1462 | 3. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001384c492 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1463 | 4. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1464 | 5. ? @ 0x00007feb32c9aa83 in ? | ||
| 1465 | 6. ? @ 0x00007feb32d2c890 in ? | ||
| 1466 | (version 23.8.7.24 (official build)) | ||
| 1467 | 2026.08.17 23:52:15.528889 [ 432271 ] {} <Information> ZooKeeperClient: Finalizing session 7. finalization_started: false, queue_finished: false, reason: 'Exception in receiveThread' | ||
| 1468 | 2026.08.17 23:52:15.528926 [ 432271 ] {} <Trace> oximeter.version (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1469 | 2026.08.17 23:52:15.528941 [ 432271 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1470 | 2026.08.17 23:52:15.528942 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> executeQuery: Query span trace_id for opentelemetry log: 45ed73b6-58e4-dff4-f2f1-4a956c5af421 | ||
| 1471 | 2026.08.17 23:52:15.528954 [ 432271 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1472 | 2026.08.17 23:52:15.528959 [ 432271 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1473 | 2026.08.17 23:52:15.528966 [ 432271 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1474 | 2026.08.17 23:52:15.528987 [ 432105 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1475 | 2026.08.17 23:52:15.529022 [ 432105 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/version/replicas/1/is_active since session has been expired | ||
| 1476 | 2026.08.17 23:52:15.529026 [ 432105 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Waiting for threads to finish | ||
| 1477 | 2026.08.17 23:52:15.529034 [ 432087 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1478 | 2026.08.17 23:52:15.529078 [ 432087 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/fields_i64_local/replicas/1/is_active since session has been expired | ||
| 1479 | 2026.08.17 23:52:15.529081 [ 432087 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Waiting for threads to finish | ||
| 1480 | 2026.08.17 23:52:15.529010 [ 432029 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1481 | 2026.08.17 23:52:15.529081 [ 432095 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1482 | 2026.08.17 23:52:15.529102 [ 432095 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/fields_uuid_local/replicas/1/is_active since session has been expired | ||
| 1483 | 2026.08.17 23:52:15.529097 [ 432029 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/measurements_cumulativef64_local/replicas/1/is_active since session has been expired | ||
| 1484 | 2026.08.17 23:52:15.529110 [ 432029 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Waiting for threads to finish | ||
| 1485 | 2026.08.17 23:52:15.529106 [ 432095 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Waiting for threads to finish | ||
| 1486 | 2026.08.17 23:52:15.529072 [ 432093 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1487 | 2026.08.17 23:52:15.529129 [ 432093 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/timeseries_schema_local/replicas/1/is_active since session has been expired | ||
| 1488 | 2026.08.17 23:52:15.529131 [ 432093 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Waiting for threads to finish | ||
| 1489 | 2026.08.17 23:52:15.529169 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> executeQuery: (from [::1]:56622) SELECT fields_i64_pivot.cpu_id AS cpu_id, fields_uuid_pivot.instance_id AS instance_id, fields_uuid_pivot.project_id AS project_id, fields_uuid_pivot.timeseries_key AS timeseries_key FROM ( SELECT timeseries_key, anyIf(field_value, field_name = 'instance_id') AS instance_id, anyIf(field_value, field_name = 'project_id') AS project_id FROM oximeter.fields_uuid WHERE timeseries_name = 'virtual_machine:cpu_busy' GROUP BY timeseries_key ) AS fields_uuid_pivot JOIN ( SELECT timeseries_key, anyIf(field_value, field_name = 'cpu_id') AS cpu_id FROM oximeter.fields_i64 WHERE timeseries_name = 'virtual_machine:cpu_busy' GROUP BY timeseries_key ) AS fields_i64_pivot ON fields_uuid_pivot.timeseries_key = fields_i64_pivot.timeseries_key (stage: Complete) | ||
| 1490 | 2026.08.17 23:52:15.529162 [ 432271 ] {} <Error> ZooKeeperClient: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below): | ||
| 1491 | |||
| 1492 | 0. Poco::Net::SocketImpl::error(int, String const&) @ 0x0000000015b5abb2 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1493 | 1. Poco::Net::SocketImpl::shutdown() @ 0x0000000015b5c39c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1494 | 2. Coordination::ZooKeeper::finalize(bool, bool, String const&) @ 0x000000001383d143 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1495 | 3. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001384e087 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1496 | 4. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1497 | 5. ? @ 0x00007feb32c9aa83 in ? | ||
| 1498 | 6. ? @ 0x00007feb32d2c890 in ? | ||
| 1499 | (version 23.8.7.24 (official build)) | ||
| 1500 | 2026.08.17 23:52:15.529180 [ 432105 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Threads finished | ||
| 1501 | 2026.08.17 23:52:15.529184 [ 432105 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 1502 | 2026.08.17 23:52:15.529193 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 1503 | 2026.08.17 23:52:15.529201 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 1504 | 2026.08.17 23:52:15.529208 [ 432087 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Threads finished | ||
| 1505 | 2026.08.17 23:52:15.529230 [ 432029 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Threads finished | ||
| 1506 | 2026.08.17 23:52:15.529253 [ 432095 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Threads finished | ||
| 1507 | 2026.08.17 23:52:15.529288 [ 432093 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Threads finished | ||
| 1508 | 2026.08.17 23:52:15.529351 [ 432273 ] {} <Information> DDLWorker: Lost ZooKeeper connection, will try to connect again: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 1509 | |||
| 1510 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1511 | 1. DB::Exception::Exception<char const (&) [16]>(int, char const (&) [16]) @ 0x000000000c63f980 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1512 | 2. Coordination::Exception Coordination::Exception::fromMessage<char const (&) [16]>(Coordination::Error, char const (&) [16]) @ 0x0000000013817dd3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1513 | 3. Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x00000000138452ec in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1514 | 4. Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x00000000138481b1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1515 | 5. zkutil::ZooKeeper::asyncTryGetChildrenNoThrow(String const&, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f728e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1516 | 6. zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f6d96 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1517 | 7. zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f7a7b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1518 | 8. zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x00000000137f75ff in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1519 | 9. DB::DDLWorker::scheduleTasks(bool) @ 0x00000000116ee6b7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1520 | 10. DB::DDLWorker::runMainThread() @ 0x00000000116e75be in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1521 | 11. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000116fd92c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1522 | 12. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1523 | 13. ? @ 0x00007feb32c9aa83 in ? | ||
| 1524 | 14. ? @ 0x00007feb32d2c890 in ? | ||
| 1525 | (version 23.8.7.24 (official build)) | ||
| 1526 | 2026.08.17 23:52:15.529431 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 1527 | 2026.08.17 23:52:15.529547 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 1528 | 2026.08.17 23:52:15.529665 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 1529 | 2026.08.17 23:52:15.529911 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 1530 | 2026.08.17 23:52:15.531320 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 1531 | 2026.08.17 23:52:15.531447 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 1532 | 2026.08.17 23:52:15.531455 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name = 'virtual_machine:cpu_busy'" moved to PREWHERE | ||
| 1533 | 2026.08.17 23:52:15.531514 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64_local | ||
| 1534 | 2026.08.17 23:52:15.531532 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1535 | 2026.08.17 23:52:15.531560 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1536 | 2026.08.17 23:52:15.531591 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> HashJoin: (0x7feaad266018) Datatype: EMPTY, kind: Inner, strictness: All, right header: fields_i64_pivot.timeseries_key UInt64 UInt64(size = 0), cpu_id Int64 Int64(size = 0) | ||
| 1537 | 2026.08.17 23:52:15.531596 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> HashJoin: (0x7feaad266018) Keys: [(timeseries_key) = (fields_i64_pivot.timeseries_key)] | ||
| 1538 | 2026.08.17 23:52:15.531780 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 1539 | 2026.08.17 23:52:15.531906 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 1540 | 2026.08.17 23:52:15.532019 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 1541 | 2026.08.17 23:52:15.532025 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name = 'virtual_machine:cpu_busy'" moved to PREWHERE | ||
| 1542 | 2026.08.17 23:52:15.532099 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid_local | ||
| 1543 | 2026.08.17 23:52:15.532116 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1544 | 2026.08.17 23:52:15.532141 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1545 | 2026.08.17 23:52:15.533287 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1546 | 2026.08.17 23:52:15.533297 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> JoiningTransform: Before join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0)' | ||
| 1547 | 2026.08.17 23:52:15.533309 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> JoiningTransform: After join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0), cpu_id Int64 Int64(size = 0)' | ||
| 1548 | 2026.08.17 23:52:15.533607 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Key condition: (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']) | ||
| 1549 | 2026.08.17 23:52:15.533662 [ 432257 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_1_1_0 (2 marks) | ||
| 1550 | 2026.08.17 23:52:15.533672 [ 432292 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 1551 | 2026.08.17 23:52:15.533679 [ 432257 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1552 | 2026.08.17 23:52:15.533683 [ 432257 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1553 | 2026.08.17 23:52:15.533687 [ 432257 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 1554 | 2026.08.17 23:52:15.533685 [ 432292 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1555 | 2026.08.17 23:52:15.533692 [ 432292 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1556 | 2026.08.17 23:52:15.533697 [ 432292 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 1557 | 2026.08.17 23:52:15.533746 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Selected 2/2 parts by partition key, 2 parts by primary key, 2/2 marks by primary key, 2 marks to read from 2 ranges | ||
| 1558 | 2026.08.17 23:52:15.533752 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 1559 | 2026.08.17 23:52:15.533784 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> MergeTreeReadPool: min_marks_for_concurrent_read=24 | ||
| 1560 | 2026.08.17 23:52:15.533794 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Reading approx. 32 rows with 2 streams | ||
| 1561 | 2026.08.17 23:52:15.533804 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1562 | 2026.08.17 23:52:15.533813 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1563 | 2026.08.17 23:52:15.533921 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Key condition: (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']) | ||
| 1564 | 2026.08.17 23:52:15.533958 [ 432294 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_1_1_0 (2 marks) | ||
| 1565 | 2026.08.17 23:52:15.533961 [ 432286 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 1566 | 2026.08.17 23:52:15.533969 [ 432294 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1567 | 2026.08.17 23:52:15.533973 [ 432294 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1568 | 2026.08.17 23:52:15.533974 [ 432286 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 1569 | 2026.08.17 23:52:15.533976 [ 432294 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 1570 | 2026.08.17 23:52:15.533978 [ 432286 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 1571 | 2026.08.17 23:52:15.533981 [ 432286 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 1572 | 2026.08.17 23:52:15.534003 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Selected 2/2 parts by partition key, 2 parts by primary key, 2/2 marks by primary key, 2 marks to read from 2 ranges | ||
| 1573 | 2026.08.17 23:52:15.534007 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 1574 | 2026.08.17 23:52:15.534029 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> MergeTreeReadPool: min_marks_for_concurrent_read=24 | ||
| 1575 | 2026.08.17 23:52:15.534034 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Reading approx. 16 rows with 2 streams | ||
| 1576 | 2026.08.17 23:52:15.534040 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1577 | 2026.08.17 23:52:15.534047 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1578 | 2026.08.17 23:52:15.534132 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> JoiningTransform: Before join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0)' | ||
| 1579 | 2026.08.17 23:52:15.534144 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> JoiningTransform: After join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0), cpu_id Int64 Int64(size = 0)' | ||
| 1580 | 2026.08.17 23:52:15.534603 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> AggregatingTransform: Aggregating | ||
| 1581 | 2026.08.17 23:52:15.534611 [ 432263 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> AggregatingTransform: Aggregating | ||
| 1582 | 2026.08.17 23:52:15.535464 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: An entry for key=1559201803930241434 found in cache: sum_of_sizes=16, median_size=8 | ||
| 1583 | 2026.08.17 23:52:15.535480 [ 432263 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: An entry for key=1559201803930241434 found in cache: sum_of_sizes=16, median_size=8 | ||
| 1584 | 2026.08.17 23:52:15.535480 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: Aggregation method: key64 | ||
| 1585 | 2026.08.17 23:52:15.535486 [ 432263 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: Aggregation method: key64 | ||
| 1586 | 2026.08.17 23:52:15.535506 [ 432263 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.001429841 sec. (5595.028 rows/sec., 92.89 KiB/sec.) | ||
| 1587 | 2026.08.17 23:52:15.535506 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.001429251 sec. (5597.337 rows/sec., 92.92 KiB/sec.) | ||
| 1588 | 2026.08.17 23:52:15.535511 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: Merging aggregated data | ||
| 1589 | 2026.08.17 23:52:15.535635 [ 432263 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> AggregatingTransform: Aggregating | ||
| 1590 | 2026.08.17 23:52:15.535638 [ 432263 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: An entry for key=11757313445726935263 found in cache: sum_of_sizes=16, median_size=8 | ||
| 1591 | 2026.08.17 23:52:15.535640 [ 432263 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: Aggregation method: key64 | ||
| 1592 | 2026.08.17 23:52:15.535648 [ 432263 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.001794791 sec. (8914.687 rows/sec., 226.35 KiB/sec.) | ||
| 1593 | 2026.08.17 23:52:15.535649 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> AggregatingTransform: Aggregating | ||
| 1594 | 2026.08.17 23:52:15.535653 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: An entry for key=11757313445726935263 found in cache: sum_of_sizes=16, median_size=8 | ||
| 1595 | 2026.08.17 23:52:15.535655 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: Aggregation method: key64 | ||
| 1596 | 2026.08.17 23:52:15.535663 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.001812841 sec. (8825.926 rows/sec., 224.10 KiB/sec.) | ||
| 1597 | 2026.08.17 23:52:15.535665 [ 432265 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> Aggregator: Merging aggregated data | ||
| 1598 | 2026.08.17 23:52:15.536019 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> executeQuery: Read 48 rows, 3.39 KiB in 0.007055 sec., 6803.685329553508 rows/sec., 480.60 KiB/sec. | ||
| 1599 | 2026.08.17 23:52:15.536124 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Trace> HashJoin: (0x7feaad266018) Join data is being destroyed, 12800 bytes and 16 rows in hash table | ||
| 1600 | 2026.08.17 23:52:15.536181 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> MemoryTracker: Peak memory usage (for query): 231.47 KiB. | ||
| 1601 | 2026.08.17 23:52:15.536183 [ 431939 ] {f2f6b18e-d261-48cd-a499-1d03ea4e4eaf} <Debug> TCPHandler: Processed in 0.008293716 sec. | ||
| 1602 | 2026.08.17 23:52:15.536431 [ 431939 ] {} <Trace> TCP-Session: 73eaf47d-a602-48f9-b745-357cd4ccad2a Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 1603 | 2026.08.17 23:52:15.536609 [ 432105 ] {} <Debug> ZooKeeperClient: Connected to a suboptimal ZooKeeper host ([::1]:19003, index 1). To preserve balance in ZooKeeper usage, this ZooKeeper session will expire in 11762 seconds | ||
| 1604 | 2026.08.17 23:52:15.537623 [ 432105 ] {} <Information> ZooKeeperClient: Connected to ZooKeeper at [::1]:19003 with session_id 9 | ||
| 1605 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 1606 | 2026.08.17 23:52:15.537664 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> executeQuery: Query span trace_id for opentelemetry log: fb26314c-23e3-90ae-5f44-b2e73dc2b21c | ||
| 1607 | 2026.08.17 23:52:15.537750 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Debug> executeQuery: (from [::1]:56622) SELECT timeseries_key, start_time, timestamp, datum FROM oximeter.measurements_cumulativef64 WHERE timeseries_name = 'virtual_machine:cpu_busy' AND ((greater(timestamp, '2000-01-01 00:00:00.000000000') AND timeseries_key IN (462795175800602962,2318161133194800099,3810615656589673070,5210923722195156639,5419464027902213652,6772052453391040098,7613129139487427710,11185254949005441257,11365707485692726787,11873430000991232922,12428098974121224233,13097606626804240593,15203798703989247109,15279762139926950523,15520859351690753739,15994013491234619255))) ORDER BY timeseries_key, start_time, timestamp LIMIT 1000001 (stage: Complete) | ||
| 1608 | 2026.08.17 23:52:15.538002 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 1609 | 2026.08.17 23:52:15.538179 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 1610 | 2026.08.17 23:52:15.538324 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 1 | ||
| 1611 | 2026.08.17 23:52:15.538335 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "(timestamp > '2000-01-01 00:00:00.000000000') AND (timeseries_key IN (462795175800602962, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6772052453391040098, 7613129139487427710, 11185254949005441257, 11365707485692726787, 11873430000991232922, 12428098974121224233, 13097606626804240593, 15203798703989247109, 15279762139926950523, 15520859351690753739, 15994013491234619255)) AND (timeseries_name = 'virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 1612 | 2026.08.17 23:52:15.538453 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64_local | ||
| 1613 | 2026.08.17 23:52:15.538478 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 1614 | 2026.08.17 23:52:15.539869 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 1615 | 2026.08.17 23:52:15.539984 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Key condition: (column 3 in ('946684800', +Inf)), (column 1 in 16-element set), and, (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']), and | ||
| 1616 | 2026.08.17 23:52:15.540025 [ 432264 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Used generic exclusion search over index for part all_0_0_0 with 1 steps | ||
| 1617 | 2026.08.17 23:52:15.540028 [ 432265 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Used generic exclusion search over index for part all_1_1_0 with 1 steps | ||
| 1618 | 2026.08.17 23:52:15.540082 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Selected 2/2 parts by partition key, 2 parts by primary key, 2/2 marks by primary key, 2 marks to read from 2 ranges | ||
| 1619 | 2026.08.17 23:52:15.540097 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Spreading ranges among streams with order | ||
| 1620 | 2026.08.17 23:52:15.540163 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 3 steps: "greater(timestamp, '2000-01-01 00:00:00.000000000')", "in(timeseries_key, (462795175800602962, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6772052453391040098, 7613129139487427710, 11185254949005441257, 11365707485692726787, 11873430000991232922, 12428098974121224233, 13097606626804240593, 15203798703989247109, 15279762139926950523, 15520859351690753739, 15994013491234619255))", "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1621 | 2026.08.17 23:52:15.540168 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_1_1_0, approx. 400 rows starting from 0 | ||
| 1622 | 2026.08.17 23:52:15.540201 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 3 steps: "greater(timestamp, '2000-01-01 00:00:00.000000000')", "in(timeseries_key, (462795175800602962, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6772052453391040098, 7613129139487427710, 11185254949005441257, 11365707485692726787, 11873430000991232922, 12428098974121224233, 13097606626804240593, 15203798703989247109, 15279762139926950523, 15520859351690753739, 15994013491234619255))", "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 1623 | 2026.08.17 23:52:15.540204 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 400 rows starting from 0 | ||
| 1624 | 2026.08.17 23:52:15.540837 [ 432293 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Debug> MergingSortedTransform: Merge sorted 1 blocks, 800 rows in 0 sec. | ||
| 1625 | 2026.08.17 23:52:15.542142 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Debug> executeQuery: Read 800 rows, 51.56 KiB in 0.004466 sec., 179131.2136139722 rows/sec., 11.27 MiB/sec. | ||
| 1626 | 2026.08.17 23:52:15.542260 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Debug> MemoryTracker: Peak memory usage (for query): 253.17 KiB. | ||
| 1627 | 2026.08.17 23:52:15.542263 [ 431939 ] {7c8d629a-cc06-4c71-bb3b-81a04ca08243} <Debug> TCPHandler: Processed in 0.005878844 sec. | ||
| 1628 | 2026.08.17 23:52:15.545420 [ 432105 ] {} <Information> ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled | ||
| 1629 | 2026.08.17 23:52:15.546417 [ 432105 ] {} <Information> ZooKeeperClient: Keeper feature flag MULTI_READ: enabled | ||
| 1630 | 2026.08.17 23:52:15.546421 [ 432105 ] {} <Information> ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: disabled | ||
| 1631 | 2026.08.17 23:52:15.546425 [ 432105 ] {} <Trace> ZooKeeper: Initialized, hosts: [::1]:19001,[::1]:19002,[::1]:19003 | ||
| 1632 | 2026.08.17 23:52:15.546430 [ 432105 ] {} <Debug> Context: Establishing a new connection with ZooKeeper took 17 ms | ||
| 1633 | 2026.08.17 23:52:15.546436 [ 432105 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1634 | 2026.08.17 23:52:15.546450 [ 432087 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1635 | 2026.08.17 23:52:15.546458 [ 432095 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1636 | 2026.08.17 23:52:15.546452 [ 432029 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1637 | 2026.08.17 23:52:15.546466 [ 432093 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 1638 | 2026.08.17 23:52:15.573152 [ 432105 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1639 | 2026.08.17 23:52:15.576232 [ 432105 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1640 | 2026.08.17 23:52:15.576249 [ 432105 ] {} <Debug> oximeter.version (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/version/replicas/1/queue | ||
| 1641 | 2026.08.17 23:52:15.576233 [ 432095 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1642 | 2026.08.17 23:52:15.576267 [ 432093 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1643 | 2026.08.17 23:52:15.576233 [ 432029 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1644 | 2026.08.17 23:52:15.576236 [ 432087 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 1645 | 2026.08.17 23:52:15.576351 [ 432105 ] {} <Debug> oximeter.version (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1646 | 2026.08.17 23:52:15.576357 [ 432093 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1647 | 2026.08.17 23:52:15.576360 [ 432029 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1648 | 2026.08.17 23:52:15.576365 [ 432093 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/timeseries_schema_local/replicas/1/queue | ||
| 1649 | 2026.08.17 23:52:15.576368 [ 432029 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/measurements_cumulativef64_local/replicas/1/queue | ||
| 1650 | 2026.08.17 23:52:15.576359 [ 432095 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1651 | 2026.08.17 23:52:15.576381 [ 432095 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/fields_uuid_local/replicas/1/queue | ||
| 1652 | 2026.08.17 23:52:15.576360 [ 432087 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 1653 | 2026.08.17 23:52:15.576389 [ 432087 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/fields_i64_local/replicas/1/queue | ||
| 1654 | 2026.08.17 23:52:15.576422 [ 432105 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1655 | 2026.08.17 23:52:15.576423 [ 432093 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1656 | 2026.08.17 23:52:15.576472 [ 432029 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1657 | 2026.08.17 23:52:15.576475 [ 432095 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1658 | 2026.08.17 23:52:15.576479 [ 432087 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 1659 | 2026.08.17 23:52:15.576499 [ 432093 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1660 | 2026.08.17 23:52:15.576543 [ 432029 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1661 | 2026.08.17 23:52:15.576547 [ 432095 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1662 | 2026.08.17 23:52:15.576551 [ 432087 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 1663 | 2026.08.17 23:52:15.577111 [ 432105 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1664 | 2026.08.17 23:52:15.577163 [ 432100 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 1665 | 2026.08.17 23:52:15.577263 [ 432093 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1666 | 2026.08.17 23:52:15.577316 [ 432029 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1667 | 2026.08.17 23:52:15.577332 [ 432034 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72963889152, merge_with_ttl_allowedtrue) | ||
| 1668 | 2026.08.17 23:52:15.577405 [ 432085 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72963889152, merge_with_ttl_allowedtrue) | ||
| 1669 | 2026.08.17 23:52:15.578593 [ 432085 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 1670 | 2026.08.17 23:52:15.577430 [ 432087 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1671 | 2026.08.17 23:52:15.577483 [ 432124 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72963889152, merge_with_ttl_allowedtrue) | ||
| 1672 | 2026.08.17 23:52:15.577539 [ 432040 ] {} <Trace> oximeter.version (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 140ms, ratio: 0, points per minute: 0) | ||
| 1673 | 2026.08.17 23:52:15.577590 [ 432095 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 1674 | 2026.08.17 23:52:15.578618 [ 432124 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 1675 | 2026.08.17 23:52:15.577785 [ 432116 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 137ms, ratio: 0, points per minute: 0) | ||
| 1676 | 2026.08.17 23:52:15.577789 [ 432094 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 137ms, ratio: 0, points per minute: 0) | ||
| 1677 | 2026.08.17 23:52:15.577806 [ 432161 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 136ms, ratio: 0, points per minute: 0) | ||
| 1678 | 2026.08.17 23:52:15.577850 [ 432145 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 137ms, ratio: 0, points per minute: 0) | ||
| 1679 | 2026.08.17 23:52:15.578582 [ 432034 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 1680 | 2026.08.17 23:52:15.577612 [ 432146 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72963889152, merge_with_ttl_allowedtrue) | ||
| 1681 | 2026.08.17 23:52:15.578663 [ 432146 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 1682 | 2026.08.17 23:52:15.593444 [ 431940 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56624 | ||
| 1683 | 2026.08.17 23:52:15.594992 [ 431940 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 1684 | 2026.08.17 23:52:15.595019 [ 431940 ] {} <Debug> TCP-Session: 7e7655f7-c1f8-4214-8bd8-bc9a7eb40e2c Authenticating user 'default' from [::1]:56624 | ||
| 1685 | 2026.08.17 23:52:15.595031 [ 431940 ] {} <Debug> TCP-Session: 7e7655f7-c1f8-4214-8bd8-bc9a7eb40e2c Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 1686 | 2026.08.17 23:52:15.595034 [ 431940 ] {} <Debug> TCP-Session: 7e7655f7-c1f8-4214-8bd8-bc9a7eb40e2c Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 1687 | 2026.08.17 23:52:15.603523 [ 432296 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56628 | ||
| 1688 | 2026.08.17 23:52:15.603918 [ 432296 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 1689 | 2026.08.17 23:52:15.603936 [ 432296 ] {} <Debug> TCP-Session: cdc55b17-cd8b-4d8b-bbdc-fec5af5684b5 Authenticating user 'default' from [::1]:56628 | ||
| 1690 | 2026.08.17 23:52:15.603952 [ 432296 ] {} <Debug> TCP-Session: cdc55b17-cd8b-4d8b-bbdc-fec5af5684b5 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 1691 | 2026.08.17 23:52:15.603955 [ 432296 ] {} <Debug> TCP-Session: cdc55b17-cd8b-4d8b-bbdc-fec5af5684b5 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 1692 | 2026.08.17 23:52:15.666476 [ 432151 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000002 - log-0000000002 | ||
| 1693 | 2026.08.17 23:52:15.681205 [ 432151 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000002 to queue with type GET_PART with virtual parts [all_2_2_0] | ||
| 1694 | 2026.08.17 23:52:15.682301 [ 432151 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 1695 | 2026.08.17 23:52:15.700726 [ 432213 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Fetching part all_2_2_0 from default:/clickhouse/tables/1/fields_i64_local/replicas/3 | ||
| 1696 | 2026.08.17 23:52:15.703719 [ 432136 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000002 - log-0000000002 | ||
| 1697 | 2026.08.17 23:52:15.703768 [ 432213 ] {} <Trace> ReadWriteBufferFromHTTP: Sending request to http://[::1]:19403/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_i64_local%2Freplicas%2F3&part=all_2_2_0&client_protocol_version=8&compress=false | ||
| 1698 | 2026.08.17 23:52:15.704110 [ 432213 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1699 | 2026.08.17 23:52:15.704123 [ 432213 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 135.90 GiB. | ||
| 1700 | 2026.08.17 23:52:15.704130 [ 432213 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Fetcher): Disk for fetch is not provided, getting disk from reservation default with type 'local' | ||
| 1701 | 2026.08.17 23:52:15.704139 [ 432213 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Fetcher): Downloading part all_2_2_0 onto disk default. | ||
| 1702 | 2026.08.17 23:52:15.704614 [ 432213 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Fetcher): Download of part all_2_2_0 onto disk default finished. | ||
| 1703 | 2026.08.17 23:52:15.706253 [ 432213 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Renaming temporary part tmp-fetch_all_2_2_0 to all_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1704 | 2026.08.17 23:52:15.706288 [ 432213 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Committing part all_2_2_0 to zookeeper | ||
| 1705 | 2026.08.17 23:52:15.707188 [ 432136 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000002 to queue with type GET_PART with virtual parts [all_2_2_0] | ||
| 1706 | 2026.08.17 23:52:15.707204 [ 432136 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 1707 | 2026.08.17 23:52:15.713703 [ 432213 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Part all_2_2_0 committed to zookeeper | ||
| 1708 | 2026.08.17 23:52:15.713735 [ 432213 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Fetched part all_2_2_0 from default:/clickhouse/tables/1/fields_i64_local/replicas/3 | ||
| 1709 | 2026.08.17 23:52:15.713806 [ 432131 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=72961945600, merge_with_ttl_allowedtrue) | ||
| 1710 | 2026.08.17 23:52:15.713826 [ 432131 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 1711 | 2026.08.17 23:52:15.720339 [ 432222 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreePartCheckThread): Enqueueing all_2_2_0 for check after after 0s | ||
| 1712 | 2026.08.17 23:52:15.720379 [ 432222 ] {} <Information> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Code: 234. DB::Exception: No active replica has part all_2_2_0 or covering part (cannot execute queue-0000000002: GET_PART with virtual parts [all_2_2_0]). (NO_REPLICA_HAS_PART) | ||
| 1713 | 2026.08.17 23:52:15.720383 [ 432128 ] {} <Information> oximeter.fields_uuid_local (ReplicatedMergeTreePartCheckThread): Checking part all_2_2_0 | ||
| 1714 | 2026.08.17 23:52:15.727971 [ 432128 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreePartCheckThread): Part all_2_2_0 in zookeeper: false, locally: false | ||
| 1715 | 2026.08.17 23:52:15.727984 [ 432128 ] {} <Information> oximeter.fields_uuid_local (ReplicatedMergeTreePartCheckThread): Checking if anyone has a part all_2_2_0 or covering part. | ||
| 1716 | 2026.08.17 23:52:15.728095 [ 432128 ] {} <Information> oximeter.fields_uuid_local (ReplicatedMergeTreePartCheckThread): Found the missing part all_2_2_0 at all_2_2_0 on 3 | ||
| 1717 | 2026.08.17 23:52:16.000383 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 394.18 MiB, peak 394.47 MiB, free memory in arenas 25.66 MiB, will set to 399.97 MiB (RSS), difference: 5.80 MiB | ||
| 1718 | 2026.08.17 23:52:16.050635 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Fetching part all_2_2_0 from default:/clickhouse/tables/1/fields_uuid_local/replicas/2 | ||
| 1719 | 2026.08.17 23:52:16.050733 [ 432209 ] {} <Trace> ReadWriteBufferFromHTTP: Sending request to http://[::1]:19402/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_uuid_local%2Freplicas%2F2&part=all_2_2_0&client_protocol_version=8&compress=false | ||
| 1720 | 2026.08.17 23:52:16.050791 [ 432209 ] {} <Trace> HTTPSessionAdapter: Created HTTP(S) session with ::1:19402 (::1:19402) | ||
| 1721 | 2026.08.17 23:52:16.052753 [ 432209 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1722 | 2026.08.17 23:52:16.052773 [ 432209 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.45 GiB. | ||
| 1723 | 2026.08.17 23:52:16.052778 [ 432209 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Disk for fetch is not provided, getting disk from reservation default with type 'local' | ||
| 1724 | 2026.08.17 23:52:16.052787 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Downloading part all_2_2_0 onto disk default. | ||
| 1725 | 2026.08.17 23:52:16.053261 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Download of part all_2_2_0 onto disk default finished. | ||
| 1726 | 2026.08.17 23:52:16.054671 [ 432209 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Renaming temporary part tmp-fetch_all_2_2_0 to all_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1727 | 2026.08.17 23:52:16.054709 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Committing part all_2_2_0 to zookeeper | ||
| 1728 | 2026.08.17 23:52:16.061912 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Part all_2_2_0 committed to zookeeper | ||
| 1729 | 2026.08.17 23:52:16.061936 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Fetched part all_2_2_0 from default:/clickhouse/tables/1/fields_uuid_local/replicas/2 | ||
| 1730 | 2026.08.17 23:52:16.061991 [ 432150 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73253410816, merge_with_ttl_allowedtrue) | ||
| 1731 | 2026.08.17 23:52:16.062005 [ 432150 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 1732 | 2026.08.17 23:52:16.529434 [ 432273 ] {} <Debug> DDLWorker: Initializing DDLWorker thread | ||
| 1733 | 2026.08.17 23:52:16.535416 [ 432273 ] {} <Debug> DDLWorker: Initialized DDLWorker thread | ||
| 1734 | 2026.08.17 23:52:16.535429 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 1735 | 2026.08.17 23:52:16.535432 [ 432273 ] {} <Trace> DDLWorker: Don't have unfinished tasks after restarting | ||
| 1736 | 2026.08.17 23:52:16.535438 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 1737 | 2026.08.17 23:52:16.535510 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=20, queue_size=20, entries=query-0000000000..query-0000000019, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=query-0000000019 | ||
| 1738 | 2026.08.17 23:52:16.535521 [ 432273 ] {} <Debug> DDLWorker: No tasks to schedule | ||
| 1739 | 2026.08.17 23:52:16.535524 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 1740 | 2026.08.17 23:52:16.831503 [ 432048 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000003 - log-0000000003 | ||
| 1741 | 2026.08.17 23:52:16.842272 [ 432048 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000003 to queue with type GET_PART with virtual parts [all_3_3_0] | ||
| 1742 | 2026.08.17 23:52:16.842292 [ 432048 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 1743 | 2026.08.17 23:52:16.858926 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Fetching part all_3_3_0 from default:/clickhouse/tables/1/fields_i64_local/replicas/3 | ||
| 1744 | 2026.08.17 23:52:16.865023 [ 432220 ] {} <Trace> ReadWriteBufferFromHTTP: Sending request to http://[::1]:19403/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_i64_local%2Freplicas%2F3&part=all_3_3_0&client_protocol_version=8&compress=false | ||
| 1745 | 2026.08.17 23:52:16.865113 [ 432021 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000003 - log-0000000003 | ||
| 1746 | 2026.08.17 23:52:16.865292 [ 432220 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1747 | 2026.08.17 23:52:16.865302 [ 432220 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.45 GiB. | ||
| 1748 | 2026.08.17 23:52:16.865306 [ 432220 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Fetcher): Disk for fetch is not provided, getting disk from reservation default with type 'local' | ||
| 1749 | 2026.08.17 23:52:16.865313 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Fetcher): Downloading part all_3_3_0 onto disk default. | ||
| 1750 | 2026.08.17 23:52:16.865717 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Fetcher): Download of part all_3_3_0 onto disk default finished. | ||
| 1751 | 2026.08.17 23:52:16.865727 [ 432220 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Renaming temporary part tmp-fetch_all_3_3_0 to all_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1752 | 2026.08.17 23:52:16.867566 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Committing part all_3_3_0 to zookeeper | ||
| 1753 | 2026.08.17 23:52:16.888306 [ 432021 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000003 to queue with type GET_PART with virtual parts [all_3_3_0] | ||
| 1754 | 2026.08.17 23:52:16.888321 [ 432021 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 1755 | 2026.08.17 23:52:16.891993 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Part all_3_3_0 committed to zookeeper | ||
| 1756 | 2026.08.17 23:52:16.892017 [ 432220 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Fetched part all_3_3_0 from default:/clickhouse/tables/1/fields_i64_local/replicas/3 | ||
| 1757 | 2026.08.17 23:52:16.892076 [ 432112 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73253814272, merge_with_ttl_allowedtrue) | ||
| 1758 | 2026.08.17 23:52:16.892085 [ 432112 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 1759 | 2026.08.17 23:52:16.895259 [ 432206 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Fetching part all_3_3_0 from default:/clickhouse/tables/1/fields_uuid_local/replicas/3 | ||
| 1760 | 2026.08.17 23:52:16.916925 [ 432206 ] {} <Trace> ReadWriteBufferFromHTTP: Sending request to http://[::1]:19403/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_uuid_local%2Freplicas%2F3&part=all_3_3_0&client_protocol_version=8&compress=false | ||
| 1761 | 2026.08.17 23:52:16.917195 [ 432206 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1762 | 2026.08.17 23:52:16.917204 [ 432206 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.45 GiB. | ||
| 1763 | 2026.08.17 23:52:16.917211 [ 432206 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Disk for fetch is not provided, getting disk from reservation default with type 'local' | ||
| 1764 | 2026.08.17 23:52:16.917219 [ 432206 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Downloading part all_3_3_0 onto disk default. | ||
| 1765 | 2026.08.17 23:52:16.919612 [ 432206 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Download of part all_3_3_0 onto disk default finished. | ||
| 1766 | 2026.08.17 23:52:16.919623 [ 432206 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Renaming temporary part tmp-fetch_all_3_3_0 to all_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1767 | 2026.08.17 23:52:16.919647 [ 432206 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Committing part all_3_3_0 to zookeeper | ||
| 1768 | 2026.08.17 23:52:16.922635 [ 432050 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000002 - log-0000000002 | ||
| 1769 | 2026.08.17 23:52:16.923167 [ 432206 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Undoing transaction (1, 1, 00000000-0000-0000-0000-000000000000). Removing parts: all_3_3_0. | ||
| 1770 | 2026.08.17 23:52:16.925401 [ 432206 ] {} <Error> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): auto DB::StorageReplicatedMergeTree::processQueueEntry(ReplicatedMergeTreeQueue::SelectedEntryPtr)::(anonymous class)::operator()(LogEntryPtr &) const: Code: 999. Coordination::Exception: Coordination error: Operation timeout. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 1771 | |||
| 1772 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1773 | 1. DB::Exception::Exception<char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type>, char const*&&) @ 0x000000000af9d01c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1774 | 2. Coordination::Exception::Exception(Coordination::Error) @ 0x00000000137efee7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1775 | 3. zkutil::ZooKeeper::tryMulti(std::vector<std::shared_ptr<Coordination::Request>, std::allocator<std::shared_ptr<Coordination::Request>>> const&, std::vector<std::shared_ptr<Coordination::Response>, std::allocator<std::shared_ptr<Coordination::Response>>>&) @ 0x00000000137fbcc2 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1776 | 4. DB::StorageReplicatedMergeTree::checkPartChecksumsAndCommit(DB::MergeTreeData::Transaction&, std::shared_ptr<DB::IMergeTreeDataPart> const&, std::optional<DB::HardlinkedFiles>, bool) @ 0x000000001279544d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1777 | 5. DB::StorageReplicatedMergeTree::fetchPart(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, String const&, String const&, bool, unsigned long, std::shared_ptr<zkutil::ZooKeeper>, bool) @ 0x00000000127be9cc in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1778 | 6. DB::StorageReplicatedMergeTree::executeFetch(DB::ReplicatedMergeTreeLogEntry&, bool) @ 0x00000000127b3310 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1779 | 7. DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x0000000012799da6 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1780 | 8. bool std::__function::__policy_invoker<bool (std::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&)>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_1, bool (std::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&)>>(std::__function::__policy_storage const*, std::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x000000001289f69f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1781 | 9. DB::ReplicatedMergeTreeQueue::processEntry(std::function<std::shared_ptr<zkutil::ZooKeeper> ()>, std::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&, std::function<bool (std::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&)>) @ 0x0000000012efc206 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1782 | 10. DB::StorageReplicatedMergeTree::processQueueEntry(std::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>) @ 0x00000000127e0c3c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1783 | 11. DB::ExecutableLambdaAdapter::executeStep() @ 0x00000000128a01f2 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1784 | 12. DB::MergeTreeBackgroundExecutor<DB::RoundRobinRuntimeQueue>::threadFunction() @ 0x0000000012be88e0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1785 | 13. ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000c7275be in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1786 | 14. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c72b07c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1787 | 15. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1788 | 16. ? @ 0x00007feb32c9aa83 in ? | ||
| 1789 | 17. ? @ 0x00007feb32d2c890 in ? | ||
| 1790 | (version 23.8.7.24 (official build)) | ||
| 1791 | 2026.08.17 23:52:17.443083 [ 432274 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' | ||
| 1792 | 2026.08.17 23:52:17.443086 [ 432269 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' | ||
| 1793 | 2026.08.17 23:52:17.443513 [ 432274 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 1794 | 2026.08.17 23:52:17.443519 [ 432269 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 1795 | 2026.08.17 23:52:17.443796 [ 432274 ] {} <Information> Application: Setting max_server_memory_usage was set to 27.67 GiB (30.74 GiB available * 0.90 max_server_memory_usage_to_ram_ratio) | ||
| 1796 | 2026.08.17 23:52:17.443799 [ 432274 ] {} <Information> Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.37 GiB (30.74 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) | ||
| 1797 | 2026.08.17 23:52:17.443801 [ 432274 ] {} <Information> Application: Merges and mutations memory limit is set to 15.37 GiB | ||
| 1798 | 2026.08.17 23:52:17.443812 [ 432269 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 1799 | 2026.08.17 23:52:17.443958 [ 432274 ] {} <Information> MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (32) for MergeMutateExecutor | ||
| 1800 | 2026.08.17 23:52:17.443963 [ 432274 ] {} <Information> MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for MoveExecutor | ||
| 1801 | 2026.08.17 23:52:17.443966 [ 432274 ] {} <Information> MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for FetchExecutor | ||
| 1802 | 2026.08.17 23:52:17.443967 [ 432274 ] {} <Information> MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for CommonExecutor | ||
| 1803 | 2026.08.17 23:52:17.443983 [ 432274 ] {} <Information> ZooKeeperClient: Finalizing session 9. finalization_started: false, queue_finished: false, reason: 'Config changed' | ||
| 1804 | 2026.08.17 23:52:17.444034 [ 432274 ] {} <Trace> oximeter.version (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1805 | 2026.08.17 23:52:17.444050 [ 432274 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1806 | 2026.08.17 23:52:17.444061 [ 432274 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1807 | 2026.08.17 23:52:17.444066 [ 432274 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1808 | 2026.08.17 23:52:17.444076 [ 432274 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Starting restating thread, schedule: true | ||
| 1809 | 2026.08.17 23:52:17.444076 [ 432076 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1810 | 2026.08.17 23:52:17.444095 [ 432076 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/measurements_cumulativef64_local/replicas/1/is_active since session has been expired | ||
| 1811 | 2026.08.17 23:52:17.444098 [ 432076 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Waiting for threads to finish | ||
| 1812 | 2026.08.17 23:52:17.444084 [ 432039 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1813 | 2026.08.17 23:52:17.444096 [ 432043 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1814 | 2026.08.17 23:52:17.444118 [ 432039 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/version/replicas/1/is_active since session has been expired | ||
| 1815 | 2026.08.17 23:52:17.444120 [ 432043 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/timeseries_schema_local/replicas/1/is_active since session has been expired | ||
| 1816 | 2026.08.17 23:52:17.444122 [ 432043 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Waiting for threads to finish | ||
| 1817 | 2026.08.17 23:52:17.444123 [ 432039 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Waiting for threads to finish | ||
| 1818 | 2026.08.17 23:52:17.444132 [ 432049 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1819 | 2026.08.17 23:52:17.444162 [ 432049 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/fields_i64_local/replicas/1/is_active since session has been expired | ||
| 1820 | 2026.08.17 23:52:17.444165 [ 432049 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Waiting for threads to finish | ||
| 1821 | 2026.08.17 23:52:17.444162 [ 432031 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): ZooKeeper session has expired. Switching to a new session. | ||
| 1822 | 2026.08.17 23:52:17.444178 [ 432031 ] {} <Debug> EphemeralNodeHolder: Cannot remove /clickhouse/tables/1/fields_uuid_local/replicas/1/is_active since session has been expired | ||
| 1823 | 2026.08.17 23:52:17.444184 [ 432031 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Waiting for threads to finish | ||
| 1824 | 2026.08.17 23:52:17.444246 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 1825 | 2026.08.17 23:52:17.444265 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 1826 | 2026.08.17 23:52:17.444282 [ 432039 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Threads finished | ||
| 1827 | 2026.08.17 23:52:17.444328 [ 432273 ] {} <Information> DDLWorker: Lost ZooKeeper connection, will try to connect again: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 1828 | |||
| 1829 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1830 | 1. DB::Exception::Exception<char const (&) [16]>(int, char const (&) [16]) @ 0x000000000c63f980 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1831 | 2. Coordination::Exception Coordination::Exception::fromMessage<char const (&) [16]>(Coordination::Error, char const (&) [16]) @ 0x0000000013817dd3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1832 | 3. Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x00000000138452ec in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1833 | 4. Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x00000000138481b1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1834 | 5. zkutil::ZooKeeper::asyncTryGetChildrenNoThrow(String const&, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f728e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1835 | 6. zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f6d96 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1836 | 7. zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f7a7b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1837 | 8. zkutil::ZooKeeper::tryGetChildren(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::ListRequestType) @ 0x00000000137f75ff in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1838 | 9. DB::DDLWorker::scheduleTasks(bool) @ 0x00000000116ee6b7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1839 | 10. DB::DDLWorker::runMainThread() @ 0x00000000116e75be in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1840 | 11. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000116fd92c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1841 | 12. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1842 | 13. ? @ 0x00007feb32c9aa83 in ? | ||
| 1843 | 14. ? @ 0x00007feb32d2c890 in ? | ||
| 1844 | (version 23.8.7.24 (official build)) | ||
| 1845 | 2026.08.17 23:52:17.444492 [ 432103 ] {} <Error> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 1846 | |||
| 1847 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1848 | 1. DB::Exception::Exception<char const (&) [16]>(int, char const (&) [16]) @ 0x000000000c63f980 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1849 | 2. Coordination::Exception Coordination::Exception::fromMessage<char const (&) [16]>(Coordination::Error, char const (&) [16]) @ 0x0000000013817dd3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1850 | 3. Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x00000000138452ec in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1851 | 4. Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000013847605 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1852 | 5. zkutil::ZooKeeper::getImpl(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x00000000137fa0bb in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1853 | 6. zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x00000000137fa7df in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1854 | 7. zkutil::ZooKeeper::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x00000000137fa6c8 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1855 | 8. zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x00000000137fa518 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1856 | 9. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000012eddcf5 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1857 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, bool, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool, DB::RenamingRestrictions, bool)::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000001289d59f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1858 | 11. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1859 | 12. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1860 | 13. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1861 | 14. ? @ 0x00007feb32c9aa83 in ? | ||
| 1862 | 15. ? @ 0x00007feb32d2c890 in ? | ||
| 1863 | (version 23.8.7.24 (official build)) | ||
| 1864 | 2026.08.17 23:52:17.444717 [ 432043 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Threads finished | ||
| 1865 | 2026.08.17 23:52:17.445392 [ 432050 ] {} <Error> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Coordination error: Connection loss. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 1866 | |||
| 1867 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1868 | 1. DB::Exception::Exception<char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type>, char const*&&) @ 0x000000000af9d01c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1869 | 2. Coordination::Exception::Exception(Coordination::Error) @ 0x00000000137efee7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1870 | 3. zkutil::KeeperMultiException::check(Coordination::Error, std::vector<std::shared_ptr<Coordination::Request>, std::allocator<std::shared_ptr<Coordination::Request>>> const&, std::vector<std::shared_ptr<Coordination::Response>, std::allocator<std::shared_ptr<Coordination::Response>>> const&) @ 0x00000000137fbc3d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1871 | 4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000012ee236f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1872 | 5. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, bool, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool, DB::RenamingRestrictions, bool)::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000001289d59f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1873 | 6. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1874 | 7. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1875 | 8. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1876 | 9. ? @ 0x00007feb32c9aa83 in ? | ||
| 1877 | 10. ? @ 0x00007feb32d2c890 in ? | ||
| 1878 | (version 23.8.7.24 (official build)) | ||
| 1879 | 2026.08.17 23:52:17.445412 [ 432050 ] {} <Trace> oximeter.measurements_cumulativef64_local (StorageReplicatedMergeTree::queueUpdatingTask): Execution took 553 ms. | ||
| 1880 | 2026.08.17 23:52:17.445495 [ 432044 ] {} <Error> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 1881 | |||
| 1882 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1883 | 1. DB::Exception::Exception<char const (&) [16]>(int, char const (&) [16]) @ 0x000000000c63f980 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1884 | 2. Coordination::Exception Coordination::Exception::fromMessage<char const (&) [16]>(Coordination::Error, char const (&) [16]) @ 0x0000000013817dd3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1885 | 3. Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x00000000138452ec in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1886 | 4. Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x00000000138481b1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1887 | 5. zkutil::ZooKeeper::asyncTryGetChildrenNoThrow(String const&, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f728e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1888 | 6. zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f6d96 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1889 | 7. zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f7a7b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1890 | 8. zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f7971 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1891 | 9. DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000012ee47c8 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1892 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, bool, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool, DB::RenamingRestrictions, bool)::$_2, void ()>>(std::__function::__policy_storage const*) @ 0x000000001289dec0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1893 | 11. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1894 | 12. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1895 | 13. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1896 | 14. ? @ 0x00007feb32c9aa83 in ? | ||
| 1897 | 15. ? @ 0x00007feb32d2c890 in ? | ||
| 1898 | (version 23.8.7.24 (official build)) | ||
| 1899 | 2026.08.17 23:52:17.445514 [ 432075 ] {} <Error> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 1900 | |||
| 1901 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1902 | 1. DB::Exception::Exception<char const (&) [16]>(int, char const (&) [16]) @ 0x000000000c63f980 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1903 | 2. Coordination::Exception Coordination::Exception::fromMessage<char const (&) [16]>(Coordination::Error, char const (&) [16]) @ 0x0000000013817dd3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1904 | 3. Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x00000000138452ec in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1905 | 4. Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x00000000138481b1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1906 | 5. zkutil::ZooKeeper::asyncTryGetChildrenNoThrow(String const&, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f728e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1907 | 6. zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f6d96 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1908 | 7. zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f7a7b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1909 | 8. zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f7971 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1910 | 9. DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000012ee47c8 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1911 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, bool, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool, DB::RenamingRestrictions, bool)::$_2, void ()>>(std::__function::__policy_storage const*) @ 0x000000001289dec0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1912 | 11. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1913 | 12. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1914 | 13. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1915 | 14. ? @ 0x00007feb32c9aa83 in ? | ||
| 1916 | 15. ? @ 0x00007feb32d2c890 in ? | ||
| 1917 | (version 23.8.7.24 (official build)) | ||
| 1918 | 2026.08.17 23:52:17.445538 [ 432023 ] {} <Error> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 1919 | |||
| 1920 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1921 | 1. DB::Exception::Exception<char const (&) [16]>(int, char const (&) [16]) @ 0x000000000c63f980 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1922 | 2. Coordination::Exception Coordination::Exception::fromMessage<char const (&) [16]>(Coordination::Error, char const (&) [16]) @ 0x0000000013817dd3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1923 | 3. Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x00000000138452ec in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1924 | 4. Coordination::ZooKeeper::get(String const&, std::function<void (Coordination::GetResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000013847605 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1925 | 5. zkutil::ZooKeeper::getImpl(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>) @ 0x00000000137fa0bb in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1926 | 6. zkutil::ZooKeeper::tryGetWatch(String const&, String&, Coordination::Stat*, std::function<void (Coordination::WatchResponse const&)>, Coordination::Error*) @ 0x00000000137fa7df in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1927 | 7. zkutil::ZooKeeper::tryGet(String const&, String&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&, Coordination::Error*) @ 0x00000000137fa6c8 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1928 | 8. zkutil::ZooKeeper::get(String const&, Coordination::Stat*, std::shared_ptr<Poco::Event> const&) @ 0x00000000137fa518 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1929 | 9. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::shared_ptr<zkutil::ZooKeeper>, std::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x0000000012eddcf5 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1930 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, bool, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool, DB::RenamingRestrictions, bool)::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000001289d59f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1931 | 11. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1932 | 12. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1933 | 13. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1934 | 14. ? @ 0x00007feb32c9aa83 in ? | ||
| 1935 | 15. ? @ 0x00007feb32d2c890 in ? | ||
| 1936 | (version 23.8.7.24 (official build)) | ||
| 1937 | 2026.08.17 23:52:17.445554 [ 432050 ] {} <Error> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 242. DB::Exception: Table is in readonly mode (replica path: /clickhouse/tables/1/measurements_cumulativef64_local/replicas/1). (TABLE_IS_READ_ONLY), Stack trace (when copying this message, always include the lines below): | ||
| 1938 | |||
| 1939 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1940 | 1. DB::Exception::Exception<String const&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type>, String const&) @ 0x00000000074d2e8d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1941 | 2. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, bool, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool, DB::RenamingRestrictions, bool)::$_1, void ()>>(std::__function::__policy_storage const*) @ 0x000000001289d6ae in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1942 | 3. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1943 | 4. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1944 | 5. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1945 | 6. ? @ 0x00007feb32c9aa83 in ? | ||
| 1946 | 7. ? @ 0x00007feb32d2c890 in ? | ||
| 1947 | (version 23.8.7.24 (official build)) | ||
| 1948 | 2026.08.17 23:52:17.445547 [ 432107 ] {} <Error> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 1949 | |||
| 1950 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1951 | 1. DB::Exception::Exception<char const (&) [16]>(int, char const (&) [16]) @ 0x000000000c63f980 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1952 | 2. Coordination::Exception Coordination::Exception::fromMessage<char const (&) [16]>(Coordination::Error, char const (&) [16]) @ 0x0000000013817dd3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1953 | 3. Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x00000000138452ec in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1954 | 4. Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function<void (Coordination::ListResponse const&)>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x00000000138481b1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1955 | 5. zkutil::ZooKeeper::asyncTryGetChildrenNoThrow(String const&, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f728e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1956 | 6. zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f6d96 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1957 | 7. zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector<String, std::allocator<String>>&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f7a7b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1958 | 8. zkutil::ZooKeeper::getChildrenWatch(String const&, Coordination::Stat*, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>, Coordination::ListRequestType) @ 0x00000000137f7971 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1959 | 9. DB::ReplicatedMergeTreeQueue::updateMutations(std::shared_ptr<zkutil::ZooKeeper>, std::shared_ptr<std::function<void (Coordination::WatchResponse const&)>>) @ 0x0000000012ee47c8 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1960 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, bool, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool, DB::RenamingRestrictions, bool)::$_2, void ()>>(std::__function::__policy_storage const*) @ 0x000000001289dec0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1961 | 11. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1962 | 12. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1963 | 13. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 1964 | 14. ? @ 0x00007feb32c9aa83 in ? | ||
| 1965 | 15. ? @ 0x00007feb32d2c890 in ? | ||
| 1966 | (version 23.8.7.24 (official build)) | ||
| 1967 | 2026.08.17 23:52:17.445626 [ 432031 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Threads finished | ||
| 1968 | 2026.08.17 23:52:17.445658 [ 432076 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Threads finished | ||
| 1969 | 2026.08.17 23:52:17.445676 [ 432049 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Threads finished | ||
| 1970 | 2026.08.17 23:52:18.000363 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 405.48 MiB, peak 410.54 MiB, free memory in arenas 35.96 MiB, will set to 410.79 MiB (RSS), difference: 5.31 MiB | ||
| 1971 | 2026.08.17 23:52:18.444407 [ 432273 ] {} <Debug> DDLWorker: Initializing DDLWorker thread | ||
| 1972 | 2026.08.17 23:52:22.843489 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 759 entries to flush up to offset 759 | ||
| 1973 | 2026.08.17 23:52:22.843490 [ 432254 ] {} <Trace> SystemLog (system.opentelemetry_span_log): Flushing system log, 120 entries to flush up to offset 120 | ||
| 1974 | 2026.08.17 23:52:22.843491 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 8 | ||
| 1975 | 2026.08.17 23:52:22.843673 [ 432255 ] {} <Debug> SystemLog (system.asynchronous_metric_log): Will use existing table system.asynchronous_metric_log for AsynchronousMetricLog | ||
| 1976 | 2026.08.17 23:52:22.843677 [ 432254 ] {} <Debug> SystemLog (system.opentelemetry_span_log): Will use existing table system.opentelemetry_span_log for OpenTelemetrySpanLog | ||
| 1977 | 2026.08.17 23:52:22.844117 [ 432254 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1978 | 2026.08.17 23:52:22.844132 [ 432254 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.90 GiB. | ||
| 1979 | 2026.08.17 23:52:22.844237 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1980 | 2026.08.17 23:52:22.844249 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.90 GiB. | ||
| 1981 | 2026.08.17 23:52:22.844459 [ 432254 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_1_1_0 (state Temporary) | ||
| 1982 | 2026.08.17 23:52:22.844648 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_1_1_0 (state Temporary) | ||
| 1983 | 2026.08.17 23:52:22.844670 [ 432254 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Renaming temporary part tmp_insert_202608_1_1_0 to 202608_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1984 | 2026.08.17 23:52:22.844762 [ 432254 ] {} <Trace> SystemLog (system.opentelemetry_span_log): Flushed system log up to offset 120 | ||
| 1985 | 2026.08.17 23:52:22.844798 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_1_1_0 to 202608_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1986 | 2026.08.17 23:52:22.844875 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 759 | ||
| 1987 | 2026.08.17 23:52:22.845033 [ 432256 ] {} <Debug> SystemLog (system.metric_log): Will use existing table system.metric_log for MetricLog | ||
| 1988 | 2026.08.17 23:52:22.852214 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 1989 | 2026.08.17 23:52:22.852233 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.90 GiB. | ||
| 1990 | 2026.08.17 23:52:22.855708 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_1_1_0 (state Temporary) | ||
| 1991 | 2026.08.17 23:52:22.858398 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_1_1_0 to 202608_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 1992 | 2026.08.17 23:52:22.859310 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 8 | ||
| 1993 | 2026.08.17 23:52:27.451623 [ 432039 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 1994 | 2026.08.17 23:52:27.452158 [ 432043 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 1995 | 2026.08.17 23:52:27.452560 [ 432031 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 1996 | 2026.08.17 23:52:27.452747 [ 432274 ] {} <Error> ConfigReloader: Error updating configuration from '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.423860.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' config.: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001 | ||
| 1997 | Code: 209. DB::NetException: Timeout exceeded while reading from socket ([::1]:19003, 10000 ms): while receiving handshake from ZooKeeper. (SOCKET_TIMEOUT) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 1998 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 1999 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2000 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2001 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2002 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2003 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2004 | |||
| 2005 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2006 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2007 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2008 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2009 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2010 | 5. zkutil::ZooKeeper::ZooKeeper(Poco::Util::AbstractConfiguration const&, String const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f66f9 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2011 | 6. DB::reloadZooKeeperIfChangedImpl(Poco::AutoPtr<Poco::Util::AbstractConfiguration> const&, String const&, std::shared_ptr<zkutil::ZooKeeper>&, std::shared_ptr<DB::ZooKeeperLog>, bool) @ 0x000000001167114c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2012 | 7. DB::Context::reloadZooKeeperIfChanged(Poco::AutoPtr<Poco::Util::AbstractConfiguration> const&) const @ 0x0000000011670f31 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2013 | 8. DB::Server::main(std::vector<String, std::allocator<String>> const&)::$_4::operator()(Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool) const @ 0x000000000c6ffe87 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2014 | 9. void std::__function::__policy_invoker<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::__call_impl<std::__function::__default_alloc_func<DB::Server::main(std::vector<String, std::allocator<String>> const&)::$_4, void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>>(std::__function::__policy_storage const*, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool) @ 0x000000000c6fda06 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2015 | 10. DB::ConfigReloader::reloadIfNewer(bool, bool, bool, bool) @ 0x00000000137cceec in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2016 | 11. DB::ConfigReloader::run() @ 0x00000000137ce1a0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2017 | 12. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::ConfigReloader::*)(), DB::ConfigReloader*>(void (DB::ConfigReloader::*&&)(), DB::ConfigReloader*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000137ce9cc in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2018 | 13. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2019 | 14. ? @ 0x00007feb32c9aa83 in ? | ||
| 2020 | 15. ? @ 0x00007feb32d2c890 in ? | ||
| 2021 | (version 23.8.7.24 (official build)) | ||
| 2022 | 2026.08.17 23:52:27.452817 [ 432039 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 2023 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2024 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2025 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2026 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2027 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2028 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2029 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2030 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2031 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2032 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2033 | |||
| 2034 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2035 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2036 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2037 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2038 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2039 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2040 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2041 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2042 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2043 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2044 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2045 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2046 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2047 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2048 | (version 23.8.7.24 (official build)) | ||
| 2049 | 2026.08.17 23:52:27.452826 [ 432043 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19003, [::1]:19002 | ||
| 2050 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2051 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2052 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2053 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2054 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2055 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2056 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2057 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2058 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2059 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2060 | |||
| 2061 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2062 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2063 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2064 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2065 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2066 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2067 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2068 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2069 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2070 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2071 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2072 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2073 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2074 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2075 | (version 23.8.7.24 (official build)) | ||
| 2076 | 2026.08.17 23:52:27.452839 [ 432039 ] {} <Trace> oximeter.version (ReplicatedMergeTreeRestartingThread): Execution took 10008 ms. | ||
| 2077 | 2026.08.17 23:52:27.452848 [ 432043 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Execution took 10008 ms. | ||
| 2078 | 2026.08.17 23:52:27.453039 [ 432076 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2079 | 2026.08.17 23:52:27.453055 [ 432031 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 2080 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2081 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2082 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2083 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2084 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2085 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2086 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2087 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2088 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2089 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2090 | |||
| 2091 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2092 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2093 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2094 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2095 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2096 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2097 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2098 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2099 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2100 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2101 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2102 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2103 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2104 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2105 | (version 23.8.7.24 (official build)) | ||
| 2106 | 2026.08.17 23:52:27.453071 [ 432031 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Execution took 10008 ms. | ||
| 2107 | 2026.08.17 23:52:27.453075 [ 432031 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2108 | 2026.08.17 23:52:27.453468 [ 432049 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2109 | 2026.08.17 23:52:27.453487 [ 432076 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 2110 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2111 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2112 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2113 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2114 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2115 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2116 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2117 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2118 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2119 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2120 | |||
| 2121 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2122 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2123 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2124 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2125 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2126 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2127 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2128 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2129 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2130 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2131 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2132 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2133 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2134 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2135 | (version 23.8.7.24 (official build)) | ||
| 2136 | 2026.08.17 23:52:27.453506 [ 432076 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Execution took 10009 ms. | ||
| 2137 | 2026.08.17 23:52:27.453915 [ 432273 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2138 | 2026.08.17 23:52:27.453948 [ 432049 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19003, [::1]:19001 | ||
| 2139 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2140 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2141 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2142 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2143 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2144 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2145 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2146 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2147 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2148 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2149 | |||
| 2150 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2151 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2152 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2153 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2154 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2155 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2156 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2157 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2158 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2159 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2160 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2161 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2162 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2163 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2164 | (version 23.8.7.24 (official build)) | ||
| 2165 | 2026.08.17 23:52:27.453966 [ 432049 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Execution took 10009 ms. | ||
| 2166 | 2026.08.17 23:52:27.453971 [ 432049 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2167 | 2026.08.17 23:52:27.454365 [ 432031 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2168 | 2026.08.17 23:52:27.454440 [ 432273 ] {} <Error> virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 2169 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2170 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2171 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2172 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2173 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2174 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2175 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2176 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2177 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2178 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2179 | |||
| 2180 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2181 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2182 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2183 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2184 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2185 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2186 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2187 | 7. DB::DDLWorker::getAndSetZooKeeper() @ 0x00000000116ec1ed in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2188 | 8. DB::DDLWorker::initializeMainThread() @ 0x00000000116fc81b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2189 | 9. DB::DDLWorker::runMainThread() @ 0x00000000116e7280 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2190 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000116fd92c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2191 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2192 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2193 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2194 | (version 23.8.7.24 (official build)) | ||
| 2195 | 2026.08.17 23:52:27.454715 [ 432049 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2196 | 2026.08.17 23:52:27.454728 [ 432031 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19003, [::1]:19002 | ||
| 2197 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2198 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2199 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2200 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2201 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2202 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2203 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2204 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2205 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2206 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2207 | |||
| 2208 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2209 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2210 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2211 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2212 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2213 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2214 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2215 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2216 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2217 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2218 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2219 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2220 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2221 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2222 | (version 23.8.7.24 (official build)) | ||
| 2223 | 2026.08.17 23:52:27.455064 [ 432049 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19002, [::1]:19003 | ||
| 2224 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2225 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2226 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2227 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2228 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2229 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2230 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2231 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2232 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2233 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2234 | |||
| 2235 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2236 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2237 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2238 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2239 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2240 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2241 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2242 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2243 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2244 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2245 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2246 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2247 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2248 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2249 | (version 23.8.7.24 (official build)) | ||
| 2250 | 2026.08.17 23:52:27.552927 [ 432098 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2251 | 2026.08.17 23:52:27.552945 [ 432018 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2252 | 2026.08.17 23:52:27.552962 [ 432098 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2253 | 2026.08.17 23:52:27.553538 [ 432018 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2254 | 2026.08.17 23:52:27.553566 [ 432098 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 2255 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2256 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2257 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2258 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2259 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2260 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2261 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2262 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2263 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2264 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2265 | |||
| 2266 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2267 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2268 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2269 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2270 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2271 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2272 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2273 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2274 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2275 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2276 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2277 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2278 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2279 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2280 | (version 23.8.7.24 (official build)) | ||
| 2281 | 2026.08.17 23:52:27.553598 [ 432117 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2282 | 2026.08.17 23:52:27.554104 [ 432117 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2283 | 2026.08.17 23:52:27.554138 [ 432018 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19002, [::1]:19003 | ||
| 2284 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2285 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2286 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2287 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2288 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2289 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2290 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2291 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2292 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2293 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2294 | |||
| 2295 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2296 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2297 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2298 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2299 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2300 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2301 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2302 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2303 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2304 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2305 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2306 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2307 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2308 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2309 | (version 23.8.7.24 (official build)) | ||
| 2310 | 2026.08.17 23:52:27.554555 [ 432117 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 2311 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2312 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2313 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2314 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2315 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2316 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2317 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2318 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2319 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2320 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2321 | |||
| 2322 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2323 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2324 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2325 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2326 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2327 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2328 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2329 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2330 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2331 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2332 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2333 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2334 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2335 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2336 | (version 23.8.7.24 (official build)) | ||
| 2337 | 2026.08.17 23:52:27.754828 [ 432119 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2338 | 2026.08.17 23:52:27.754853 [ 432119 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2339 | 2026.08.17 23:52:27.755177 [ 432118 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2340 | 2026.08.17 23:52:27.755580 [ 432118 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2341 | 2026.08.17 23:52:27.755612 [ 432119 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 2342 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2343 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2344 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2345 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2346 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2347 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2348 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2349 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2350 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2351 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2352 | |||
| 2353 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2354 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2355 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2356 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2357 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2358 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2359 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2360 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2361 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2362 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2363 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2364 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2365 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2366 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2367 | (version 23.8.7.24 (official build)) | ||
| 2368 | 2026.08.17 23:52:27.756090 [ 432118 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 2369 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2370 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2371 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2372 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2373 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2374 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2375 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2376 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2377 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2378 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2379 | |||
| 2380 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2381 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2382 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2383 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2384 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2385 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2386 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2387 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2388 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2389 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2390 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2391 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2392 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2393 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2394 | (version 23.8.7.24 (official build)) | ||
| 2395 | 2026.08.17 23:52:27.853682 [ 432143 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2396 | 2026.08.17 23:52:27.853706 [ 432143 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2397 | 2026.08.17 23:52:27.854221 [ 432143 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19003, [::1]:19001 | ||
| 2398 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2399 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2400 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2401 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2402 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2403 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2404 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2405 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2406 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2407 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2408 | |||
| 2409 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2410 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2411 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2412 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2413 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2414 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2415 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2416 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2417 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2418 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2419 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2420 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2421 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2422 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2423 | (version 23.8.7.24 (official build)) | ||
| 2424 | 2026.08.17 23:52:27.854250 [ 432133 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2425 | 2026.08.17 23:52:27.854262 [ 432133 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2426 | 2026.08.17 23:52:27.854643 [ 432156 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2427 | 2026.08.17 23:52:27.854709 [ 432156 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2428 | 2026.08.17 23:52:27.854730 [ 432133 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19002, [::1]:19003 | ||
| 2429 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2430 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2431 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2432 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2433 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2434 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2435 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2436 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2437 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2438 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2439 | |||
| 2440 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2441 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2442 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2443 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2444 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2445 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2446 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2447 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2448 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2449 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2450 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2451 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2452 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2453 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2454 | (version 23.8.7.24 (official build)) | ||
| 2455 | 2026.08.17 23:52:27.855167 [ 432156 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 2456 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2457 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2458 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2459 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2460 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2461 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2462 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2463 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2464 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2465 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2466 | |||
| 2467 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2468 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2469 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2470 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2471 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2472 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2473 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2474 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2475 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2476 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2477 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2478 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2479 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2480 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2481 | (version 23.8.7.24 (official build)) | ||
| 2482 | 2026.08.17 23:52:28.000342 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 415.43 MiB, peak 423.94 MiB, free memory in arenas 71.45 MiB, will set to 422.64 MiB (RSS), difference: 7.21 MiB | ||
| 2483 | 2026.08.17 23:52:28.355720 [ 432129 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2484 | 2026.08.17 23:52:28.355746 [ 432129 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2485 | 2026.08.17 23:52:28.356202 [ 432126 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2486 | 2026.08.17 23:52:28.356400 [ 432126 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2487 | 2026.08.17 23:52:28.356440 [ 432129 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 2488 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2489 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2490 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2491 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2492 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2493 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2494 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2495 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2496 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2497 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2498 | |||
| 2499 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2500 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2501 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2502 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2503 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2504 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2505 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2506 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2507 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2508 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2509 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2510 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2511 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2512 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2513 | (version 23.8.7.24 (official build)) | ||
| 2514 | 2026.08.17 23:52:28.356990 [ 432126 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19003, [::1]:19001 | ||
| 2515 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2516 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2517 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2518 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2519 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2520 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2521 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2522 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2523 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2524 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2525 | |||
| 2526 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2527 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2528 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2529 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2530 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2531 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2532 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2533 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2534 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2535 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2536 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2537 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2538 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2539 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2540 | (version 23.8.7.24 (official build)) | ||
| 2541 | 2026.08.17 23:52:28.454322 [ 432139 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2542 | 2026.08.17 23:52:28.454345 [ 432139 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2543 | 2026.08.17 23:52:28.454822 [ 432083 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2544 | 2026.08.17 23:52:28.454879 [ 432083 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2545 | 2026.08.17 23:52:28.454905 [ 432139 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 2546 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2547 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2548 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2549 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2550 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2551 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2552 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2553 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2554 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2555 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2556 | |||
| 2557 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2558 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2559 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2560 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2561 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2562 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2563 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2564 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2565 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2566 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2567 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2568 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2569 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2570 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2571 | (version 23.8.7.24 (official build)) | ||
| 2572 | 2026.08.17 23:52:28.455252 [ 432084 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2573 | 2026.08.17 23:52:28.455366 [ 432084 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2574 | 2026.08.17 23:52:28.455386 [ 432083 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19003, [::1]:19001 | ||
| 2575 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2576 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2577 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2578 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2579 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2580 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2581 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2582 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2583 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2584 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2585 | |||
| 2586 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2587 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2588 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2589 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2590 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2591 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2592 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2593 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2594 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2595 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2596 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2597 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2598 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2599 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2600 | (version 23.8.7.24 (official build)) | ||
| 2601 | 2026.08.17 23:52:28.455843 [ 432084 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19003, [::1]:19002 | ||
| 2602 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2603 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2604 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2605 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2606 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2607 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2608 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2609 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2610 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2611 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2612 | |||
| 2613 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2614 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2615 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2616 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2617 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2618 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2619 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2620 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2621 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2622 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2623 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2624 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2625 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2626 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2627 | (version 23.8.7.24 (official build)) | ||
| 2628 | 2026.08.17 23:52:29.356541 [ 432121 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2629 | 2026.08.17 23:52:29.356569 [ 432121 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2630 | 2026.08.17 23:52:29.357085 [ 432122 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2631 | 2026.08.17 23:52:29.357265 [ 432122 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2632 | 2026.08.17 23:52:29.357292 [ 432121 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19003, [::1]:19002 | ||
| 2633 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2634 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2635 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2636 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2637 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2638 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2639 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2640 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2641 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2642 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2643 | |||
| 2644 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2645 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2646 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2647 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2648 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2649 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2650 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2651 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2652 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2653 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2654 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2655 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2656 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2657 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2658 | (version 23.8.7.24 (official build)) | ||
| 2659 | 2026.08.17 23:52:29.357763 [ 432122 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 2660 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2661 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2662 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2663 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2664 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2665 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2666 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2667 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2668 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2669 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2670 | |||
| 2671 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2672 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2673 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2674 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2675 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2676 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2677 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2678 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2679 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2680 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2681 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2682 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2683 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2684 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2685 | (version 23.8.7.24 (official build)) | ||
| 2686 | 2026.08.17 23:52:29.455001 [ 432105 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2687 | 2026.08.17 23:52:29.455015 [ 432105 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2688 | 2026.08.17 23:52:29.455476 [ 432100 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2689 | 2026.08.17 23:52:29.455623 [ 432100 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2690 | 2026.08.17 23:52:29.455649 [ 432105 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 2691 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2692 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2693 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2694 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2695 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2696 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2697 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2698 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2699 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2700 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2701 | |||
| 2702 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2703 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2704 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2705 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2706 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2707 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2708 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2709 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2710 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2711 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2712 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2713 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2714 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2715 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2716 | (version 23.8.7.24 (official build)) | ||
| 2717 | 2026.08.17 23:52:29.455920 [ 432093 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2718 | 2026.08.17 23:52:29.456079 [ 432093 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2719 | 2026.08.17 23:52:29.456107 [ 432100 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19003, [::1]:19001 | ||
| 2720 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2721 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2722 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2723 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2724 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2725 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2726 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2727 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2728 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2729 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2730 | |||
| 2731 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2732 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2733 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2734 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2735 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2736 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2737 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2738 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2739 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2740 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2741 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2742 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2743 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2744 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2745 | (version 23.8.7.24 (official build)) | ||
| 2746 | 2026.08.17 23:52:29.456497 [ 432093 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 2747 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2748 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2749 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2750 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2751 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2752 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2753 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2754 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2755 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2756 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2757 | |||
| 2758 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2759 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2760 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2761 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2762 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2763 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2764 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2765 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2766 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2767 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2768 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2769 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2770 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2771 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2772 | (version 23.8.7.24 (official build)) | ||
| 2773 | 2026.08.17 23:52:30.000321 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 426.74 MiB, peak 426.74 MiB, free memory in arenas 74.25 MiB, will set to 422.72 MiB (RSS), difference: -4.02 MiB | ||
| 2774 | 2026.08.17 23:52:30.325563 [ 432087 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 2775 | 2026.08.17 23:52:30.325591 [ 432087 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 2776 | 2026.08.17 23:52:30.344948 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 1288 entries to flush up to offset 2047 | ||
| 2777 | 2026.08.17 23:52:30.345625 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 2778 | 2026.08.17 23:52:30.345638 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 137.03 GiB. | ||
| 2779 | 2026.08.17 23:52:30.346021 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_2_2_0 (state Temporary) | ||
| 2780 | 2026.08.17 23:52:30.346172 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_2_2_0 to 202608_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 2781 | 2026.08.17 23:52:30.346257 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 2047 | ||
| 2782 | 2026.08.17 23:52:30.359384 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 16 | ||
| 2783 | 2026.08.17 23:52:30.366114 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 2784 | 2026.08.17 23:52:30.366132 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 137.03 GiB. | ||
| 2785 | 2026.08.17 23:52:30.371031 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_2_2_0 (state Temporary) | ||
| 2786 | 2026.08.17 23:52:30.373651 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_2_2_0 to 202608_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 2787 | 2026.08.17 23:52:30.374574 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 16 | ||
| 2788 | 2026.08.17 23:52:30.857393 [ 432116 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2789 | 2026.08.17 23:52:30.857407 [ 432116 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2790 | 2026.08.17 23:52:30.857848 [ 432094 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2791 | 2026.08.17 23:52:30.858073 [ 432094 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2792 | 2026.08.17 23:52:30.858101 [ 432116 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19003, [::1]:19002 | ||
| 2793 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2794 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2795 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2796 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2797 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2798 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2799 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2800 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2801 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2802 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2803 | |||
| 2804 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2805 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2806 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2807 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2808 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2809 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2810 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2811 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2812 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2813 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2814 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2815 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2816 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2817 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2818 | (version 23.8.7.24 (official build)) | ||
| 2819 | 2026.08.17 23:52:30.858626 [ 432094 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19003, [::1]:19002 | ||
| 2820 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2821 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2822 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2823 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2824 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2825 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2826 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2827 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2828 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2829 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2830 | |||
| 2831 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2832 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2833 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2834 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2835 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2836 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2837 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2838 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2839 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2840 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2841 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2842 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2843 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2844 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2845 | (version 23.8.7.24 (official build)) | ||
| 2846 | 2026.08.17 23:52:30.955740 [ 432161 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2847 | 2026.08.17 23:52:30.955752 [ 432161 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2848 | 2026.08.17 23:52:30.956195 [ 432145 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2849 | 2026.08.17 23:52:30.956324 [ 432145 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2850 | 2026.08.17 23:52:30.956363 [ 432161 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 2851 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2852 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2853 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2854 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2855 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2856 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2857 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2858 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2859 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2860 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2861 | |||
| 2862 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2863 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2864 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2865 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2866 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2867 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2868 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2869 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2870 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2871 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2872 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2873 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2874 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2875 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2876 | (version 23.8.7.24 (official build)) | ||
| 2877 | 2026.08.17 23:52:30.956585 [ 432034 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2878 | 2026.08.17 23:52:30.956790 [ 432034 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2879 | 2026.08.17 23:52:30.956822 [ 432145 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19003, [::1]:19002 | ||
| 2880 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2881 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2882 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2883 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2884 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2885 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2886 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2887 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2888 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2889 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2890 | |||
| 2891 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2892 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2893 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2894 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2895 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2896 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2897 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2898 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2899 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2900 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2901 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2902 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2903 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2904 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2905 | (version 23.8.7.24 (official build)) | ||
| 2906 | 2026.08.17 23:52:30.957274 [ 432034 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19002, [::1]:19003 | ||
| 2907 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2908 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2909 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2910 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2911 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2912 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2913 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2914 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2915 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2916 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2917 | |||
| 2918 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2919 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2920 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2921 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2922 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2923 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2924 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2925 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2926 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2927 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2928 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2929 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2930 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2931 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2932 | (version 23.8.7.24 (official build)) | ||
| 2933 | 2026.08.17 23:52:31.000269 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 422.46 MiB, peak 431.32 MiB, free memory in arenas 82.04 MiB, will set to 425.56 MiB (RSS), difference: 3.10 MiB | ||
| 2934 | 2026.08.17 23:52:32.454525 [ 432273 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2935 | 2026.08.17 23:52:32.456676 [ 432273 ] {} <Error> virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19002, [::1]:19003 | ||
| 2936 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2937 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2938 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2939 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2940 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2941 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2942 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2943 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2944 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2945 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2946 | |||
| 2947 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2948 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2949 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2950 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2951 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2952 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2953 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2954 | 7. DB::DDLWorker::getAndSetZooKeeper() @ 0x00000000116ec1ed in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2955 | 8. DB::DDLWorker::initializeMainThread() @ 0x00000000116fc81b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2956 | 9. DB::DDLWorker::runMainThread() @ 0x00000000116e7280 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2957 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000116fd92c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2958 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2959 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2960 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2961 | (version 23.8.7.24 (official build)) | ||
| 2962 | 2026.08.17 23:52:32.958197 [ 432131 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2963 | 2026.08.17 23:52:32.958215 [ 432131 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2964 | 2026.08.17 23:52:32.958734 [ 432160 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 2965 | 2026.08.17 23:52:32.958757 [ 432160 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 2966 | 2026.08.17 23:52:32.958757 [ 432131 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 2967 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2968 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2969 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2970 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2971 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2972 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2973 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2974 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2975 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2976 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 2977 | |||
| 2978 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2979 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2980 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2981 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2982 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2983 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2984 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2985 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2986 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2987 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2988 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2989 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 2990 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 2991 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 2992 | (version 23.8.7.24 (official build)) | ||
| 2993 | 2026.08.17 23:52:32.959229 [ 432160 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 2994 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2995 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2996 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 2997 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 2998 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 2999 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3000 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3001 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3002 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3003 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3004 | |||
| 3005 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3006 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3007 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3008 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3009 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3010 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3011 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3012 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3013 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3014 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3015 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3016 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3017 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3018 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3019 | (version 23.8.7.24 (official build)) | ||
| 3020 | 2026.08.17 23:52:33.056461 [ 432128 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3021 | 2026.08.17 23:52:33.056477 [ 432128 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3022 | 2026.08.17 23:52:33.056928 [ 432127 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3023 | 2026.08.17 23:52:33.057139 [ 432127 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3024 | 2026.08.17 23:52:33.057168 [ 432128 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 3025 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3026 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3027 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3028 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3029 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3030 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3031 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3032 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3033 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3034 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3035 | |||
| 3036 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3037 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3038 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3039 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3040 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3041 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3042 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3043 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3044 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3045 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3046 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3047 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3048 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3049 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3050 | (version 23.8.7.24 (official build)) | ||
| 3051 | 2026.08.17 23:52:33.057360 [ 432137 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3052 | 2026.08.17 23:52:33.057592 [ 432137 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3053 | 2026.08.17 23:52:33.057623 [ 432127 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19002, [::1]:19003 | ||
| 3054 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3055 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3056 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3057 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3058 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3059 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3060 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3061 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3062 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3063 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3064 | |||
| 3065 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3066 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3067 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3068 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3069 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3070 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3071 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3072 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3073 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3074 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3075 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3076 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3077 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3078 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3079 | (version 23.8.7.24 (official build)) | ||
| 3080 | 2026.08.17 23:52:33.058040 [ 432137 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19002, [::1]:19003 | ||
| 3081 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3082 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3083 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3084 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3085 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3086 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3087 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3088 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3089 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3090 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3091 | |||
| 3092 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3093 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3094 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3095 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3096 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3097 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3098 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3099 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3100 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3101 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3102 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3103 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3104 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3105 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3106 | (version 23.8.7.24 (official build)) | ||
| 3107 | 2026.08.17 23:52:35.758867 [ 432140 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3108 | 2026.08.17 23:52:35.758894 [ 432140 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3109 | 2026.08.17 23:52:35.759318 [ 432120 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3110 | 2026.08.17 23:52:35.759456 [ 432120 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3111 | 2026.08.17 23:52:35.759485 [ 432140 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 3112 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3113 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3114 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3115 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3116 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3117 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3118 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3119 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3120 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3121 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3122 | |||
| 3123 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3124 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3125 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3126 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3127 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3128 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3129 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3130 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3131 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3132 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3133 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3134 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3135 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3136 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3137 | (version 23.8.7.24 (official build)) | ||
| 3138 | 2026.08.17 23:52:35.759974 [ 432120 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19003, [::1]:19001 | ||
| 3139 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3140 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3141 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3142 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3143 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3144 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3145 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3146 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3147 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3148 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3149 | |||
| 3150 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3151 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3152 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3153 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3154 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3155 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3156 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3157 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3158 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3159 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3160 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3161 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3162 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3163 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3164 | (version 23.8.7.24 (official build)) | ||
| 3165 | 2026.08.17 23:52:35.857266 [ 432157 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3166 | 2026.08.17 23:52:35.857292 [ 432157 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3167 | 2026.08.17 23:52:35.857704 [ 432141 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3168 | 2026.08.17 23:52:35.857854 [ 432141 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3169 | 2026.08.17 23:52:35.857877 [ 432157 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 3170 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3171 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3172 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3173 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3174 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3175 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3176 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3177 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3178 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3179 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3180 | |||
| 3181 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3182 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3183 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3184 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3185 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3186 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3187 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3188 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3189 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3190 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3191 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3192 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3193 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3194 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3195 | (version 23.8.7.24 (official build)) | ||
| 3196 | 2026.08.17 23:52:35.858122 [ 432142 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3197 | 2026.08.17 23:52:35.858287 [ 432142 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3198 | 2026.08.17 23:52:35.858313 [ 432141 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 3199 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3200 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3201 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3202 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3203 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3204 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3205 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3206 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3207 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3208 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3209 | |||
| 3210 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3211 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3212 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3213 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3214 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3215 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3216 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3217 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3218 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3219 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3220 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3221 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3222 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3223 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3224 | (version 23.8.7.24 (official build)) | ||
| 3225 | 2026.08.17 23:52:35.858736 [ 432142 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 3226 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3227 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3228 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3229 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3230 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3231 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3232 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3233 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3234 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3235 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3236 | |||
| 3237 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3238 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3239 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3240 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3241 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3242 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3243 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3244 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3245 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3246 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3247 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3248 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3249 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3250 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3251 | (version 23.8.7.24 (official build)) | ||
| 3252 | 2026.08.17 23:52:36.054655 [ 433547 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56642 | ||
| 3253 | 2026.08.17 23:52:36.054699 [ 433547 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3254 | 2026.08.17 23:52:36.054716 [ 433547 ] {} <Debug> TCP-Session: 018c82de-14b4-469e-a47a-20d706b2ef32 Authenticating user 'default' from [::1]:56642 | ||
| 3255 | 2026.08.17 23:52:36.054726 [ 433547 ] {} <Debug> TCP-Session: 018c82de-14b4-469e-a47a-20d706b2ef32 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3256 | 2026.08.17 23:52:36.054729 [ 433547 ] {} <Debug> TCP-Session: 018c82de-14b4-469e-a47a-20d706b2ef32 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3257 | 2026.08.17 23:52:36.056976 [ 433548 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56644 | ||
| 3258 | 2026.08.17 23:52:36.057015 [ 433548 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3259 | 2026.08.17 23:52:36.057025 [ 433548 ] {} <Debug> TCP-Session: b366c33f-609b-4f5c-9389-f8f56ce41be2 Authenticating user 'default' from [::1]:56644 | ||
| 3260 | 2026.08.17 23:52:36.057031 [ 433548 ] {} <Debug> TCP-Session: b366c33f-609b-4f5c-9389-f8f56ce41be2 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3261 | 2026.08.17 23:52:36.057033 [ 433548 ] {} <Debug> TCP-Session: b366c33f-609b-4f5c-9389-f8f56ce41be2 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3262 | 2026.08.17 23:52:36.061753 [ 433549 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56646 | ||
| 3263 | 2026.08.17 23:52:36.061800 [ 433549 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3264 | 2026.08.17 23:52:36.061812 [ 433549 ] {} <Debug> TCP-Session: c3ed44b9-1a2c-4998-bbb4-6d5bedfb202c Authenticating user 'default' from [::1]:56646 | ||
| 3265 | 2026.08.17 23:52:36.061822 [ 433549 ] {} <Debug> TCP-Session: c3ed44b9-1a2c-4998-bbb4-6d5bedfb202c Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3266 | 2026.08.17 23:52:36.061825 [ 433549 ] {} <Debug> TCP-Session: c3ed44b9-1a2c-4998-bbb4-6d5bedfb202c Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3267 | 2026.08.17 23:52:36.068634 [ 433550 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56648 | ||
| 3268 | 2026.08.17 23:52:36.068675 [ 433550 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3269 | 2026.08.17 23:52:36.068687 [ 433550 ] {} <Debug> TCP-Session: 8b607243-f237-4e29-8c35-2960b97e0698 Authenticating user 'default' from [::1]:56648 | ||
| 3270 | 2026.08.17 23:52:36.068692 [ 433550 ] {} <Debug> TCP-Session: 8b607243-f237-4e29-8c35-2960b97e0698 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3271 | 2026.08.17 23:52:36.068694 [ 433550 ] {} <Debug> TCP-Session: 8b607243-f237-4e29-8c35-2960b97e0698 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3272 | 2026.08.17 23:52:37.456762 [ 432273 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3273 | 2026.08.17 23:52:37.457296 [ 432273 ] {} <Error> virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 3274 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3275 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3276 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3277 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3278 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3279 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3280 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3281 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3282 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3283 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3284 | |||
| 3285 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3286 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3287 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3288 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3289 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3290 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3291 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3292 | 7. DB::DDLWorker::getAndSetZooKeeper() @ 0x00000000116ec1ed in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3293 | 8. DB::DDLWorker::initializeMainThread() @ 0x00000000116fc81b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3294 | 9. DB::DDLWorker::runMainThread() @ 0x00000000116e7280 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3295 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000116fd92c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3296 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3297 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3298 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3299 | (version 23.8.7.24 (official build)) | ||
| 3300 | 2026.08.17 23:52:37.846339 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 4301 | ||
| 3301 | 2026.08.17 23:52:37.847278 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3302 | 2026.08.17 23:52:37.847294 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 137.01 GiB. | ||
| 3303 | 2026.08.17 23:52:37.847720 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_3_3_0 (state Temporary) | ||
| 3304 | 2026.08.17 23:52:37.847890 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_3_3_0 to 202608_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3305 | 2026.08.17 23:52:37.847978 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 4301 | ||
| 3306 | 2026.08.17 23:52:37.874646 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 23 | ||
| 3307 | 2026.08.17 23:52:37.881864 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3308 | 2026.08.17 23:52:37.881881 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 137.01 GiB. | ||
| 3309 | 2026.08.17 23:52:37.885324 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_3_3_0 (state Temporary) | ||
| 3310 | 2026.08.17 23:52:37.887878 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_3_3_0 to 202608_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3311 | 2026.08.17 23:52:37.888705 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 23 | ||
| 3312 | 2026.08.17 23:52:38.000368 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 426.25 MiB, peak 434.28 MiB, free memory in arenas 75.74 MiB, will set to 427.55 MiB (RSS), difference: 1.30 MiB | ||
| 3313 | 2026.08.17 23:52:39.359593 [ 432077 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3314 | 2026.08.17 23:52:39.359619 [ 432077 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3315 | 2026.08.17 23:52:39.360089 [ 432082 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3316 | 2026.08.17 23:52:39.360175 [ 432082 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3317 | 2026.08.17 23:52:39.360193 [ 432077 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 3318 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3319 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3320 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3321 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3322 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3323 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3324 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3325 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3326 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3327 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3328 | |||
| 3329 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3330 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3331 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3332 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3333 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3334 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3335 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3336 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3337 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3338 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3339 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3340 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3341 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3342 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3343 | (version 23.8.7.24 (official build)) | ||
| 3344 | 2026.08.17 23:52:39.360724 [ 432082 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 3345 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3346 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3347 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3348 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3349 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3350 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3351 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3352 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3353 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3354 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3355 | |||
| 3356 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3357 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3358 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3359 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3360 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3361 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3362 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3363 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3364 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3365 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3366 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3367 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3368 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3369 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3370 | (version 23.8.7.24 (official build)) | ||
| 3371 | 2026.08.17 23:52:39.457963 [ 432112 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3372 | 2026.08.17 23:52:39.457976 [ 432112 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3373 | 2026.08.17 23:52:39.458393 [ 432091 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3374 | 2026.08.17 23:52:39.458462 [ 432091 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3375 | 2026.08.17 23:52:39.458486 [ 432112 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19002, [::1]:19003 | ||
| 3376 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3377 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3378 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3379 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3380 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3381 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3382 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3383 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3384 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3385 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3386 | |||
| 3387 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3388 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3389 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3390 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3391 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3392 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3393 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3394 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3395 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3396 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3397 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3398 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3399 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3400 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3401 | (version 23.8.7.24 (official build)) | ||
| 3402 | 2026.08.17 23:52:39.458815 [ 432059 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3403 | 2026.08.17 23:52:39.458910 [ 432059 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3404 | 2026.08.17 23:52:39.458934 [ 432091 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 3405 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3406 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3407 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3408 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3409 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3410 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3411 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3412 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3413 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3414 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3415 | |||
| 3416 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3417 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3418 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3419 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3420 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3421 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3422 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3423 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3424 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3425 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3426 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3427 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3428 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3429 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3430 | (version 23.8.7.24 (official build)) | ||
| 3431 | 2026.08.17 23:52:39.459320 [ 432059 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19001, [::1]:19002 | ||
| 3432 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3433 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3434 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3435 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3436 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3437 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3438 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3439 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3440 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3441 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3442 | |||
| 3443 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3444 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3445 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3446 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3447 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3448 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3449 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3450 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3451 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3452 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3453 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3454 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3455 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3456 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3457 | (version 23.8.7.24 (official build)) | ||
| 3458 | 2026.08.17 23:52:42.457389 [ 432273 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3459 | 2026.08.17 23:52:42.457971 [ 432273 ] {} <Error> virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 3460 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3461 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3462 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3463 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3464 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3465 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3466 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3467 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3468 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3469 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3470 | |||
| 3471 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3472 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3473 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3474 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3475 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3476 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3477 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3478 | 7. DB::DDLWorker::getAndSetZooKeeper() @ 0x00000000116ec1ed in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3479 | 8. DB::DDLWorker::initializeMainThread() @ 0x00000000116fc81b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3480 | 9. DB::DDLWorker::runMainThread() @ 0x00000000116e7280 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3481 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000116fd92c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3482 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3483 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3484 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3485 | (version 23.8.7.24 (official build)) | ||
| 3486 | 2026.08.17 23:52:43.860272 [ 432107 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3487 | 2026.08.17 23:52:43.860296 [ 432107 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3488 | 2026.08.17 23:52:43.860807 [ 432057 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3489 | 2026.08.17 23:52:43.860870 [ 432057 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3490 | 2026.08.17 23:52:43.860883 [ 432107 ] {} <Error> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19002, [::1]:19003 | ||
| 3491 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3492 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3493 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3494 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3495 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3496 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3497 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3498 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3499 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3500 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3501 | |||
| 3502 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3503 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3504 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3505 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3506 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3507 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3508 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3509 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3510 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3511 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3512 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3513 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3514 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3515 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3516 | (version 23.8.7.24 (official build)) | ||
| 3517 | 2026.08.17 23:52:43.861333 [ 432057 ] {} <Error> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19003, [::1]:19002 | ||
| 3518 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3519 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3520 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3521 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3522 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3523 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3524 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3525 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3526 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3527 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3528 | |||
| 3529 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3530 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3531 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3532 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3533 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3534 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3535 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3536 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3537 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3538 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3539 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3540 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3541 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3542 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3543 | (version 23.8.7.24 (official build)) | ||
| 3544 | 2026.08.17 23:52:43.958574 [ 432102 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3545 | 2026.08.17 23:52:43.958595 [ 432102 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3546 | 2026.08.17 23:52:43.959030 [ 432032 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3547 | 2026.08.17 23:52:43.959178 [ 432032 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3548 | 2026.08.17 23:52:43.959208 [ 432102 ] {} <Error> oximeter.version (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19003, [::1]:19002, [::1]:19001 | ||
| 3549 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3550 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3551 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3552 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3553 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3554 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3555 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3556 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3557 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3558 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3559 | |||
| 3560 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3561 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3562 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3563 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3564 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3565 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3566 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3567 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3568 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3569 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3570 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3571 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3572 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3573 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3574 | (version 23.8.7.24 (official build)) | ||
| 3575 | 2026.08.17 23:52:43.959397 [ 432106 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3576 | 2026.08.17 23:52:43.959669 [ 432106 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3577 | 2026.08.17 23:52:43.959689 [ 432032 ] {} <Error> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19001, [::1]:19003, [::1]:19002 | ||
| 3578 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3579 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3580 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3581 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3582 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3583 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3584 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3585 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3586 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3587 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3588 | |||
| 3589 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3590 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3591 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3592 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3593 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3594 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3595 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3596 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3597 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3598 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3599 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3600 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3601 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3602 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3603 | (version 23.8.7.24 (official build)) | ||
| 3604 | 2026.08.17 23:52:43.960120 [ 432106 ] {} <Error> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Failed to establish a new ZK connection. Will try again: Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 3605 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3606 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3607 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3608 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3609 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3610 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3611 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3612 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3613 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3614 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3615 | |||
| 3616 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3617 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3618 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3619 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3620 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3621 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3622 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3623 | 7. DB::StorageReplicatedMergeTree::setZooKeeper() @ 0x000000001275b85e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3624 | 8. DB::ReplicatedMergeTreeRestartingThread::run() @ 0x0000000012f11640 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3625 | 9. DB::BackgroundSchedulePool::threadFunction() @ 0x00000000110c859f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3626 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, char const*)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000110c95d1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3627 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3628 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3629 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3630 | (version 23.8.7.24 (official build)) | ||
| 3631 | 2026.08.17 23:52:45.325664 [ 432022 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 3632 | 2026.08.17 23:52:45.325681 [ 432022 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 3633 | 2026.08.17 23:52:45.348054 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 6877 | ||
| 3634 | 2026.08.17 23:52:45.349133 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3635 | 2026.08.17 23:52:45.349150 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.99 GiB. | ||
| 3636 | 2026.08.17 23:52:45.349524 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_4_4_0 (state Temporary) | ||
| 3637 | 2026.08.17 23:52:45.349659 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_4_4_0 to 202608_5_5_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3638 | 2026.08.17 23:52:45.349744 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 6877 | ||
| 3639 | 2026.08.17 23:52:45.388777 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 31 | ||
| 3640 | 2026.08.17 23:52:45.396771 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3641 | 2026.08.17 23:52:45.396788 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.99 GiB. | ||
| 3642 | 2026.08.17 23:52:45.400314 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_4_4_0 (state Temporary) | ||
| 3643 | 2026.08.17 23:52:45.402897 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_4_4_0 to 202608_5_5_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3644 | 2026.08.17 23:52:45.403736 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 31 | ||
| 3645 | 2026.08.17 23:52:46.000376 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 428.79 MiB, peak 436.82 MiB, free memory in arenas 75.04 MiB, will set to 427.71 MiB (RSS), difference: -1.07 MiB | ||
| 3646 | 2026.08.17 23:52:47.458065 [ 432273 ] {} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3647 | 2026.08.17 23:52:47.458621 [ 432273 ] {} <Error> virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:19002, [::1]:19001, [::1]:19003 | ||
| 3648 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3649 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3650 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3651 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3652 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3653 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3654 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19002 | ||
| 3655 | Poco::Exception. Code: 1000, e.code() = 111, Connection refused (version 23.8.7.24 (official build)), [::1]:19001 | ||
| 3656 | Code: 999. Coordination::Exception: Keeper server rejected the connection during the handshake. Possibly it's overloaded, doesn't see leader or stale: while receiving handshake from ZooKeeper. (KEEPER_EXCEPTION) (version 23.8.7.24 (official build)), [::1]:19003 | ||
| 3657 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 3658 | |||
| 3659 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3660 | 1. DB::Exception::Exception<String&>(int, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000074d294d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3661 | 2. Coordination::Exception::Exception<String&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<String&>::type>, String&) @ 0x00000000126eb50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3662 | 3. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&) @ 0x0000000013841f00 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3663 | 4. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x00000000137f31d3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3664 | 5. zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x00000000137f5ff0 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3665 | 6. DB::Context::getZooKeeper() const @ 0x000000001166b47c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3666 | 7. DB::DDLWorker::getAndSetZooKeeper() @ 0x00000000116ec1ed in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3667 | 8. DB::DDLWorker::initializeMainThread() @ 0x00000000116fc81b in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3668 | 9. DB::DDLWorker::runMainThread() @ 0x00000000116e7280 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3669 | 10. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000116fd92c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3670 | 11. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c7298c4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 3671 | 12. ? @ 0x00007feb32c9aa83 in ? | ||
| 3672 | 13. ? @ 0x00007feb32d2c890 in ? | ||
| 3673 | (version 23.8.7.24 (official build)) | ||
| 3674 | 2026.08.17 23:52:47.668279 [ 432296 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 3675 | 2026.08.17 23:52:47.668296 [ 432296 ] {} <Debug> TCP-Session: cdc55b17-cd8b-4d8b-bbdc-fec5af5684b5 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3676 | 2026.08.17 23:52:47.668313 [ 431938 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 3677 | 2026.08.17 23:52:47.668316 [ 431940 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 3678 | 2026.08.17 23:52:47.668329 [ 431939 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 3679 | 2026.08.17 23:52:47.668335 [ 431938 ] {} <Debug> TCP-Session: f8dc188c-ca4e-4008-be73-58346a54e0e4 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3680 | 2026.08.17 23:52:47.668335 [ 431940 ] {} <Debug> TCP-Session: 7e7655f7-c1f8-4214-8bd8-bc9a7eb40e2c Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3681 | 2026.08.17 23:52:47.668339 [ 433549 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 3682 | 2026.08.17 23:52:47.668347 [ 431939 ] {} <Debug> TCP-Session: 73eaf47d-a602-48f9-b745-357cd4ccad2a Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3683 | 2026.08.17 23:52:47.668345 [ 433547 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 3684 | 2026.08.17 23:52:47.668364 [ 433549 ] {} <Debug> TCP-Session: c3ed44b9-1a2c-4998-bbb4-6d5bedfb202c Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3685 | 2026.08.17 23:52:47.668358 [ 433550 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 3686 | 2026.08.17 23:52:47.668372 [ 433548 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 3687 | 2026.08.17 23:52:47.668379 [ 433547 ] {} <Debug> TCP-Session: 018c82de-14b4-469e-a47a-20d706b2ef32 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3688 | 2026.08.17 23:52:47.668387 [ 433548 ] {} <Debug> TCP-Session: b366c33f-609b-4f5c-9389-f8f56ce41be2 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3689 | 2026.08.17 23:52:47.668382 [ 433550 ] {} <Debug> TCP-Session: 8b607243-f237-4e29-8c35-2960b97e0698 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3690 | 2026.08.17 23:52:47.687635 [ 433548 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56652 | ||
| 3691 | 2026.08.17 23:52:47.687676 [ 433548 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3692 | 2026.08.17 23:52:47.687685 [ 433548 ] {} <Debug> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Authenticating user 'default' from [::1]:56652 | ||
| 3693 | 2026.08.17 23:52:47.687695 [ 433548 ] {} <Debug> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3694 | 2026.08.17 23:52:47.687699 [ 433548 ] {} <Debug> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3695 | 2026.08.17 23:52:47.687920 [ 432296 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56660 | ||
| 3696 | 2026.08.17 23:52:47.687937 [ 433547 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56662 | ||
| 3697 | 2026.08.17 23:52:47.687952 [ 431939 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56656 | ||
| 3698 | 2026.08.17 23:52:47.687959 [ 433550 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56666 | ||
| 3699 | 2026.08.17 23:52:47.687955 [ 431938 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56654 | ||
| 3700 | 2026.08.17 23:52:47.687956 [ 431940 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56658 | ||
| 3701 | 2026.08.17 23:52:47.687956 [ 433549 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56664 | ||
| 3702 | 2026.08.17 23:52:47.688041 [ 431938 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3703 | 2026.08.17 23:52:47.688055 [ 431938 ] {} <Debug> TCP-Session: 609225df-4b76-40ae-9458-3310c59c57d9 Authenticating user 'default' from [::1]:56654 | ||
| 3704 | 2026.08.17 23:52:47.688064 [ 431939 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3705 | 2026.08.17 23:52:47.688064 [ 433559 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56668 | ||
| 3706 | 2026.08.17 23:52:47.688071 [ 431939 ] {} <Debug> TCP-Session: ecccb2a5-5540-46a9-8941-3402becfae5d Authenticating user 'default' from [::1]:56656 | ||
| 3707 | 2026.08.17 23:52:47.688069 [ 431940 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3708 | 2026.08.17 23:52:47.688084 [ 432296 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3709 | 2026.08.17 23:52:47.688065 [ 431938 ] {} <Debug> TCP-Session: 609225df-4b76-40ae-9458-3310c59c57d9 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3710 | 2026.08.17 23:52:47.688088 [ 433547 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3711 | 2026.08.17 23:52:47.688092 [ 431940 ] {} <Debug> TCP-Session: a09377a6-5278-42c3-be39-a1d8738232bd Authenticating user 'default' from [::1]:56658 | ||
| 3712 | 2026.08.17 23:52:47.688101 [ 433549 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3713 | 2026.08.17 23:52:47.688108 [ 431940 ] {} <Debug> TCP-Session: a09377a6-5278-42c3-be39-a1d8738232bd Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3714 | 2026.08.17 23:52:47.688093 [ 432296 ] {} <Debug> TCP-Session: 0af081e4-339a-49c1-9e68-910979dcec70 Authenticating user 'default' from [::1]:56660 | ||
| 3715 | 2026.08.17 23:52:47.688112 [ 431940 ] {} <Debug> TCP-Session: a09377a6-5278-42c3-be39-a1d8738232bd Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3716 | 2026.08.17 23:52:47.688099 [ 431938 ] {} <Debug> TCP-Session: 609225df-4b76-40ae-9458-3310c59c57d9 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3717 | 2026.08.17 23:52:47.688081 [ 431939 ] {} <Debug> TCP-Session: ecccb2a5-5540-46a9-8941-3402becfae5d Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3718 | 2026.08.17 23:52:47.688130 [ 431939 ] {} <Debug> TCP-Session: ecccb2a5-5540-46a9-8941-3402becfae5d Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3719 | 2026.08.17 23:52:47.688115 [ 433550 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3720 | 2026.08.17 23:52:47.688142 [ 433550 ] {} <Debug> TCP-Session: 4c0f0ad7-c046-4308-85fa-f07c2377a6c4 Authenticating user 'default' from [::1]:56666 | ||
| 3721 | 2026.08.17 23:52:47.688100 [ 433547 ] {} <Debug> TCP-Session: abaefab2-85c9-435d-b18d-07599f065526 Authenticating user 'default' from [::1]:56662 | ||
| 3722 | 2026.08.17 23:52:47.688148 [ 433550 ] {} <Debug> TCP-Session: 4c0f0ad7-c046-4308-85fa-f07c2377a6c4 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3723 | 2026.08.17 23:52:47.688151 [ 433550 ] {} <Debug> TCP-Session: 4c0f0ad7-c046-4308-85fa-f07c2377a6c4 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3724 | 2026.08.17 23:52:47.688116 [ 432296 ] {} <Debug> TCP-Session: 0af081e4-339a-49c1-9e68-910979dcec70 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3725 | 2026.08.17 23:52:47.688159 [ 432296 ] {} <Debug> TCP-Session: 0af081e4-339a-49c1-9e68-910979dcec70 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3726 | 2026.08.17 23:52:47.688122 [ 433559 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 3727 | 2026.08.17 23:52:47.688151 [ 433547 ] {} <Debug> TCP-Session: abaefab2-85c9-435d-b18d-07599f065526 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3728 | 2026.08.17 23:52:47.688173 [ 433559 ] {} <Debug> TCP-Session: 754374cb-46d0-4c7c-b0d0-ba38fa24ffe6 Authenticating user 'default' from [::1]:56668 | ||
| 3729 | 2026.08.17 23:52:47.688108 [ 433549 ] {} <Debug> TCP-Session: 0763c507-088e-4eec-976b-357b45555ce7 Authenticating user 'default' from [::1]:56664 | ||
| 3730 | 2026.08.17 23:52:47.688174 [ 433547 ] {} <Debug> TCP-Session: abaefab2-85c9-435d-b18d-07599f065526 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3731 | 2026.08.17 23:52:47.688186 [ 433549 ] {} <Debug> TCP-Session: 0763c507-088e-4eec-976b-357b45555ce7 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3732 | 2026.08.17 23:52:47.688189 [ 433549 ] {} <Debug> TCP-Session: 0763c507-088e-4eec-976b-357b45555ce7 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3733 | 2026.08.17 23:52:47.688192 [ 433559 ] {} <Debug> TCP-Session: 754374cb-46d0-4c7c-b0d0-ba38fa24ffe6 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3734 | 2026.08.17 23:52:47.688196 [ 433559 ] {} <Debug> TCP-Session: 754374cb-46d0-4c7c-b0d0-ba38fa24ffe6 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3735 | 2026.08.17 23:52:47.688416 [ 431939 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 3736 | 2026.08.17 23:52:47.688430 [ 431939 ] {} <Debug> TCP-Session: ecccb2a5-5540-46a9-8941-3402becfae5d Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 3737 | 2026.08.17 23:52:48.000391 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 423.63 MiB, peak 436.82 MiB, free memory in arenas 81.24 MiB, will set to 427.93 MiB (RSS), difference: 4.30 MiB | ||
| 3738 | 2026.08.17 23:52:48.359749 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 3739 | 2026.08.17 23:52:48.359856 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Trace> executeQuery: Query span trace_id for opentelemetry log: 129329e1-3e39-428f-1fdc-af0961e322b9 | ||
| 3740 | 2026.08.17 23:52:48.359918 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Debug> executeQuery: (from [::1]:56652) /* We purposefully split our DB schema into two *disjoint* files: | ||
| 3741 | * `db-init-1.sql` and `db-init-2.sql`. The purpose of this split is to shorten | ||
| 3742 | * the duration of our replicated tests. These tests only use a subset of the | ||
| 3743 | * tables defined in the full schema. We put the tables used by the replicated | ||
| 3744 | * tests in `db-init-1.sql`, and the remainder of the tables in `db-init-2.sql`. | ||
| 3745 | * This minimizes test time by reducing the cost to load a schema. In | ||
| 3746 | * production, we load `db-init-1.sql` followed by `db-init-2.sql` so we have | ||
| 3747 | * the full schema. If we end up needing to use more tables in replicated tests | ||
| 3748 | * we can go ahead and move them into `db-init-1.sql`, removing them from | ||
| 3749 | * `db-init-2.sql`. Conversely, if we stop using given tables in our tests we | ||
| 3750 | * can move them from `db-init-1.sql` into `db-init-2.sql` and keep our test | ||
| 3751 | * times minimal. | ||
| 3752 | |||
| 3753 | * The reason to keep the two files disjoint is so that we don't have to | ||
| 3754 | * maintain consistency between table definitions. All tables are defined | ||
| 3755 | * once. However, in order to write a test that ensures the tables are in fact | ||
| 3756 | * disjoint, we must create the `oximeter` database in both files so we can load | ||
| 3757 | * them in isolation. | ||
| 3758 | */ | ||
| 3759 | CREATE DATABASE IF NOT EXISTS oximeter ON CLUSTER oximeter_cluster (stage: Complete) | ||
| 3760 | 2026.08.17 23:52:48.359981 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Trace> ContextAccess (default): Access granted: CREATE DATABASE ON oximeter.* | ||
| 3761 | 2026.08.17 23:52:48.360007 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Debug> Context: Trying to establish a new connection with ZooKeeper | ||
| 3762 | 2026.08.17 23:52:48.363283 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Information> ZooKeeperClient: Connected to ZooKeeper at [::1]:19001 with session_id 15 | ||
| 3763 | 2026.08.17 23:52:48.363392 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Information> ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled | ||
| 3764 | 2026.08.17 23:52:48.363394 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Information> ZooKeeperClient: Keeper feature flag MULTI_READ: enabled | ||
| 3765 | 2026.08.17 23:52:48.363396 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Information> ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: disabled | ||
| 3766 | 2026.08.17 23:52:48.363399 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Trace> ZooKeeper: Initialized, hosts: [::1]:19001,[::1]:19002,[::1]:19003 | ||
| 3767 | 2026.08.17 23:52:48.363404 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Debug> Context: Establishing a new connection with ZooKeeper took 3 ms | ||
| 3768 | 2026.08.17 23:52:49.000373 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 431.99 MiB, peak 436.82 MiB, free memory in arenas 81.21 MiB, will set to 428.22 MiB (RSS), difference: -3.77 MiB | ||
| 3769 | 2026.08.17 23:52:49.360999 [ 432099 ] {} <Warning> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3770 | 2026.08.17 23:52:49.361019 [ 432099 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 3771 | 2026.08.17 23:52:49.361416 [ 432069 ] {} <Warning> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3772 | 2026.08.17 23:52:49.361436 [ 432069 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 3773 | 2026.08.17 23:52:49.367838 [ 432099 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 3774 | 2026.08.17 23:52:49.370923 [ 432099 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 3775 | 2026.08.17 23:52:49.370930 [ 432099 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/fields_uuid_local/replicas/1/queue | ||
| 3776 | 2026.08.17 23:52:49.374052 [ 432099 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Having 1 queue entries to load, 0 entries already loaded. | ||
| 3777 | 2026.08.17 23:52:49.374105 [ 432069 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 3778 | 2026.08.17 23:52:49.374131 [ 432099 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000003 to queue with type GET_PART with virtual parts [all_3_3_0] | ||
| 3779 | 2026.08.17 23:52:49.374165 [ 432069 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 3780 | 2026.08.17 23:52:49.374169 [ 432069 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/fields_i64_local/replicas/1/queue | ||
| 3781 | 2026.08.17 23:52:49.374181 [ 432099 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 3782 | 2026.08.17 23:52:49.374217 [ 432069 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 3783 | 2026.08.17 23:52:49.374253 [ 432069 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 3784 | 2026.08.17 23:52:49.374477 [ 432099 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 3785 | 2026.08.17 23:52:49.374504 [ 432081 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73438515200, merge_with_ttl_allowedtrue) | ||
| 3786 | 2026.08.17 23:52:49.374500 [ 432045 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Found 1 old parts to remove. Parts: [all_3_3_0] | ||
| 3787 | 2026.08.17 23:52:49.374525 [ 432045 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removing 1 old parts from ZooKeeper | ||
| 3788 | 2026.08.17 23:52:49.374518 [ 432081 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 3789 | 2026.08.17 23:52:49.374599 [ 432045 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): There is no part all_3_3_0 in ZooKeeper, it was only in filesystem | ||
| 3790 | 2026.08.17 23:52:49.374602 [ 432045 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removed 1 old parts from ZooKeeper. Removing them from filesystem. | ||
| 3791 | 2026.08.17 23:52:49.374606 [ 432045 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removing 1 parts from filesystem (serially): Parts: [all_3_3_0] | ||
| 3792 | 2026.08.17 23:52:49.374644 [ 432069 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 3793 | 2026.08.17 23:52:49.374656 [ 432026 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73438515200, merge_with_ttl_allowedtrue) | ||
| 3794 | 2026.08.17 23:52:49.374662 [ 432026 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 3795 | 2026.08.17 23:52:49.374712 [ 432213 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Part all_3_3_0 should be deleted after previous attempt before fetch | ||
| 3796 | 2026.08.17 23:52:49.374774 [ 432045 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removing 1 parts from memory: Parts: [all_3_3_0] | ||
| 3797 | 2026.08.17 23:52:49.374778 [ 432045 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removed 1 old parts | ||
| 3798 | 2026.08.17 23:52:49.374785 [ 432033 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeCleanupThread): Checking 4 blocks (2 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/fields_i64_local/blocks | ||
| 3799 | 2026.08.17 23:52:49.374889 [ 432033 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 33797ms, ratio: 0, points per minute: 0) | ||
| 3800 | 2026.08.17 23:52:49.374894 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Fetching part all_3_3_0 from default:/clickhouse/tables/1/fields_uuid_local/replicas/3 | ||
| 3801 | 2026.08.17 23:52:49.374931 [ 432045 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeCleanupThread): Checking 4 blocks (2 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/fields_uuid_local/blocks | ||
| 3802 | 2026.08.17 23:52:49.374963 [ 432209 ] {} <Trace> ReadWriteBufferFromHTTP: Sending request to http://[::1]:19403/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_uuid_local%2Freplicas%2F3&part=all_3_3_0&client_protocol_version=8&compress=false | ||
| 3803 | 2026.08.17 23:52:49.375005 [ 432045 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0.25641027, interval: 33797ms, ratio: 0.0017094018, points per minute: 0.45520657) | ||
| 3804 | 2026.08.17 23:52:49.375149 [ 432045 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 0ms, ratio: 0, points per minute: -nan) | ||
| 3805 | 2026.08.17 23:52:49.375303 [ 432209 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3806 | 2026.08.17 23:52:49.375314 [ 432209 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 3807 | 2026.08.17 23:52:49.375318 [ 432209 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Disk for fetch is not provided, getting disk from reservation default with type 'local' | ||
| 3808 | 2026.08.17 23:52:49.375326 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Downloading part all_3_3_0 onto disk default. | ||
| 3809 | 2026.08.17 23:52:49.375808 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Fetcher): Download of part all_3_3_0 onto disk default finished. | ||
| 3810 | 2026.08.17 23:52:49.375821 [ 432209 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Renaming temporary part tmp-fetch_all_3_3_0 to all_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3811 | 2026.08.17 23:52:49.375844 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Committing part all_3_3_0 to zookeeper | ||
| 3812 | 2026.08.17 23:52:49.379442 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Part all_3_3_0 committed to zookeeper | ||
| 3813 | 2026.08.17 23:52:49.379463 [ 432209 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Fetched part all_3_3_0 from default:/clickhouse/tables/1/fields_uuid_local/replicas/3 | ||
| 3814 | 2026.08.17 23:52:49.379496 [ 432064 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73438502912, merge_with_ttl_allowedtrue) | ||
| 3815 | 2026.08.17 23:52:49.379507 [ 432064 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 3816 | 2026.08.17 23:52:49.459320 [ 432043 ] {} <Warning> oximeter.version (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3817 | 2026.08.17 23:52:49.459333 [ 432043 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 3818 | 2026.08.17 23:52:49.459778 [ 432090 ] {} <Warning> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3819 | 2026.08.17 23:52:49.459790 [ 432090 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 3820 | 2026.08.17 23:52:49.460211 [ 432076 ] {} <Warning> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table was in readonly mode. Will try to activate it. | ||
| 3821 | 2026.08.17 23:52:49.460236 [ 432076 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Trying to start replica up | ||
| 3822 | 2026.08.17 23:52:49.468997 [ 432043 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 3823 | 2026.08.17 23:52:49.475087 [ 432043 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Queue initialized | ||
| 3824 | 2026.08.17 23:52:49.475091 [ 432043 ] {} <Debug> oximeter.version (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/version/replicas/1/queue | ||
| 3825 | 2026.08.17 23:52:49.478062 [ 432076 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 3826 | 2026.08.17 23:52:49.478066 [ 432043 ] {} <Debug> oximeter.version (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 3827 | 2026.08.17 23:52:49.478106 [ 432076 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 3828 | 2026.08.17 23:52:49.478109 [ 432043 ] {} <Trace> oximeter.version (ReplicatedMergeTreeQueue): Loaded queue | ||
| 3829 | 2026.08.17 23:52:49.478110 [ 432076 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/measurements_cumulativef64_local/replicas/1/queue | ||
| 3830 | 2026.08.17 23:52:49.478125 [ 432090 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Initializing parts in queue | ||
| 3831 | 2026.08.17 23:52:49.478145 [ 432076 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 3832 | 2026.08.17 23:52:49.478157 [ 432090 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Queue initialized | ||
| 3833 | 2026.08.17 23:52:49.478161 [ 432090 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Loading queue from /clickhouse/tables/1/timeseries_schema_local/replicas/1/queue | ||
| 3834 | 2026.08.17 23:52:49.478194 [ 432076 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 3835 | 2026.08.17 23:52:49.478198 [ 432090 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Having 0 queue entries to load, 0 entries already loaded. | ||
| 3836 | 2026.08.17 23:52:49.478242 [ 432090 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeQueue): Loaded queue | ||
| 3837 | 2026.08.17 23:52:49.478359 [ 432076 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000002 - log-0000000002 | ||
| 3838 | 2026.08.17 23:52:49.478368 [ 432043 ] {} <Debug> oximeter.version (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 3839 | 2026.08.17 23:52:49.478377 [ 432018 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 3840 | 2026.08.17 23:52:49.478448 [ 432090 ] {} <Debug> oximeter.timeseries_schema_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 3841 | 2026.08.17 23:52:49.478469 [ 432133 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73438435328, merge_with_ttl_allowedtrue) | ||
| 3842 | 2026.08.17 23:52:49.478479 [ 432133 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 3843 | 2026.08.17 23:52:49.481556 [ 432076 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000002 to queue with type GET_PART with virtual parts [all_3_3_0] | ||
| 3844 | 2026.08.17 23:52:49.481561 [ 432076 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 3845 | 2026.08.17 23:52:49.481639 [ 432020 ] {} <Trace> oximeter.version (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 33904ms, ratio: 0, points per minute: 0) | ||
| 3846 | 2026.08.17 23:52:49.481647 [ 432123 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 33904ms, ratio: 0, points per minute: 0) | ||
| 3847 | 2026.08.17 23:52:49.481681 [ 432076 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeRestartingThread): Table started successfully | ||
| 3848 | 2026.08.17 23:52:49.481694 [ 432083 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73438427136, merge_with_ttl_allowedtrue) | ||
| 3849 | 2026.08.17 23:52:49.481712 [ 432083 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 3850 | 2026.08.17 23:52:49.481800 [ 432220 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Fetching part all_3_3_0 from default:/clickhouse/tables/1/measurements_cumulativef64_local/replicas/3 | ||
| 3851 | 2026.08.17 23:52:49.481810 [ 432084 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeCleanupThread): Checking 3 blocks (1 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/measurements_cumulativef64_local/blocks | ||
| 3852 | 2026.08.17 23:52:49.481854 [ 432220 ] {} <Trace> ReadWriteBufferFromHTTP: Sending request to http://[::1]:19403/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Fmeasurements_cumulativef64_local%2Freplicas%2F3&part=all_3_3_0&client_protocol_version=8&compress=false | ||
| 3853 | 2026.08.17 23:52:49.481882 [ 432084 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 33904ms, ratio: 0, points per minute: 0) | ||
| 3854 | 2026.08.17 23:52:49.482057 [ 432220 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3855 | 2026.08.17 23:52:49.482064 [ 432220 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 3856 | 2026.08.17 23:52:49.482066 [ 432220 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Fetcher): Disk for fetch is not provided, getting disk from reservation default with type 'local' | ||
| 3857 | 2026.08.17 23:52:49.482071 [ 432220 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Fetcher): Downloading part all_3_3_0 onto disk default. | ||
| 3858 | 2026.08.17 23:52:49.482457 [ 432220 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Fetcher): Download of part all_3_3_0 onto disk default finished. | ||
| 3859 | 2026.08.17 23:52:49.482465 [ 432220 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Renaming temporary part tmp-fetch_all_3_3_0 to all_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3860 | 2026.08.17 23:52:49.482483 [ 432220 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Committing part all_3_3_0 to zookeeper | ||
| 3861 | 2026.08.17 23:52:49.485849 [ 432220 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Part all_3_3_0 committed to zookeeper | ||
| 3862 | 2026.08.17 23:52:49.485866 [ 432220 ] {} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Fetched part all_3_3_0 from default:/clickhouse/tables/1/measurements_cumulativef64_local/replicas/3 | ||
| 3863 | 2026.08.17 23:52:49.485890 [ 432153 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73438392320, merge_with_ttl_allowedtrue) | ||
| 3864 | 2026.08.17 23:52:49.485905 [ 432153 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 3865 | 2026.08.17 23:52:52.462179 [ 432273 ] {} <Debug> DDLWorker: Initialized DDLWorker thread | ||
| 3866 | 2026.08.17 23:52:52.462193 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 3867 | 2026.08.17 23:52:52.462196 [ 432273 ] {} <Trace> DDLWorker: Don't have unfinished tasks after restarting | ||
| 3868 | 2026.08.17 23:52:52.462206 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 3869 | 2026.08.17 23:52:52.462316 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=21, queue_size=21, entries=query-0000000000..query-0000000020, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=query-0000000019 | ||
| 3870 | 2026.08.17 23:52:52.462326 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000020 | ||
| 3871 | 2026.08.17 23:52:52.462330 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000020 | ||
| 3872 | 2026.08.17 23:52:52.462557 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000020 (CREATE DATABASE IF NOT EXISTS oximeter UUID '15508b34-d1c9-4a0a-a3d5-a44fcdaaa6b2') | ||
| 3873 | 2026.08.17 23:52:52.465732 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE DATABASE IF NOT EXISTS oximeter UUID '15508b34-d1c9-4a0a-a3d5-a44fcdaaa6b2' | ||
| 3874 | 2026.08.17 23:52:52.465805 [ 432273 ] {53644ddb-6f1a-43cc-bea9-070546f82d13} <Trace> executeQuery: Query span trace_id for opentelemetry log: 129329e1-3e39-428f-1fdc-af0961e322b9 | ||
| 3875 | 2026.08.17 23:52:52.465820 [ 432273 ] {53644ddb-6f1a-43cc-bea9-070546f82d13} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 680aa279-d059-461a-9f5e-60b39f1237ea) /* ddl_entry=query-0000000020 */ CREATE DATABASE IF NOT EXISTS oximeter UUID '15508b34-d1c9-4a0a-a3d5-a44fcdaaa6b2' (stage: Complete) | ||
| 3876 | 2026.08.17 23:52:52.465884 [ 432273 ] {53644ddb-6f1a-43cc-bea9-070546f82d13} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000020 */ CREATE DATABASE IF NOT EXISTS oximeter UUID '15508b34-d1c9-4a0a-a3d5-a44fcdaaa6b2' | ||
| 3877 | 2026.08.17 23:52:52.469120 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 3878 | 2026.08.17 23:52:52.845066 [ 432254 ] {} <Trace> SystemLog (system.opentelemetry_span_log): Flushing system log, 5 entries to flush up to offset 125 | ||
| 3879 | 2026.08.17 23:52:52.845466 [ 432254 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3880 | 2026.08.17 23:52:52.845482 [ 432254 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 3881 | 2026.08.17 23:52:52.845797 [ 432254 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_2_2_0 (state Temporary) | ||
| 3882 | 2026.08.17 23:52:52.845974 [ 432254 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Renaming temporary part tmp_insert_202608_2_2_0 to 202608_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3883 | 2026.08.17 23:52:52.846077 [ 432254 ] {} <Trace> SystemLog (system.opentelemetry_span_log): Flushed system log up to offset 125 | ||
| 3884 | 2026.08.17 23:52:52.849833 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2256 entries to flush up to offset 9133 | ||
| 3885 | 2026.08.17 23:52:52.851067 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3886 | 2026.08.17 23:52:52.851081 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 3887 | 2026.08.17 23:52:52.851534 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_5_5_0 (state Temporary) | ||
| 3888 | 2026.08.17 23:52:52.851653 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_5_5_0 to 202608_6_6_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3889 | 2026.08.17 23:52:52.851718 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 9133 | ||
| 3890 | 2026.08.17 23:52:52.851734 [ 432135 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_1_0 to 202608_6_6_0 | ||
| 3891 | 2026.08.17 23:52:52.851753 [ 432135 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3892 | 2026.08.17 23:52:52.851758 [ 432135 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 3893 | 2026.08.17 23:52:52.851907 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_1_0 to 202608_6_6_0 into Compact with storage Full | ||
| 3894 | 2026.08.17 23:52:52.851978 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 3895 | 2026.08.17 23:52:52.852001 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_1_0, total 2360 rows starting from the beginning of the part | ||
| 3896 | 2026.08.17 23:52:52.852068 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_2_2_0, total 759 rows starting from the beginning of the part | ||
| 3897 | 2026.08.17 23:52:52.852092 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_3_3_0, total 1288 rows starting from the beginning of the part | ||
| 3898 | 2026.08.17 23:52:52.852108 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_4_4_0, total 2254 rows starting from the beginning of the part | ||
| 3899 | 2026.08.17 23:52:52.852127 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_5_5_0, total 2576 rows starting from the beginning of the part | ||
| 3900 | 2026.08.17 23:52:52.852148 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_6_6_0, total 2256 rows starting from the beginning of the part | ||
| 3901 | 2026.08.17 23:52:52.854241 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 11493 rows, containing 4 columns (4 merged, 0 gathered) in 0.002403672 sec., 4781434.4053598 rows/sec., 72.96 MiB/sec. | ||
| 3902 | 2026.08.17 23:52:52.854412 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Trace> MergedBlockOutputStream: filled checksums 202608_1_6_1 (state Temporary) | ||
| 3903 | 2026.08.17 23:52:52.854560 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_6_1 to 202608_1_6_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3904 | 2026.08.17 23:52:52.854588 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_6_1} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_1_0, 202608_6_6_0] -> 202608_1_6_1 | ||
| 3905 | 2026.08.17 23:52:52.854631 [ 432197 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.87 MiB. | ||
| 3906 | 2026.08.17 23:52:52.903821 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 38 | ||
| 3907 | 2026.08.17 23:52:52.911887 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3908 | 2026.08.17 23:52:52.911902 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 3909 | 2026.08.17 23:52:52.915290 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_5_5_0 (state Temporary) | ||
| 3910 | 2026.08.17 23:52:52.917845 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_5_5_0 to 202608_6_6_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3911 | 2026.08.17 23:52:52.917990 [ 432028 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_1_0 to 202608_6_6_0 | ||
| 3912 | 2026.08.17 23:52:52.918012 [ 432028 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 3913 | 2026.08.17 23:52:52.918018 [ 432028 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 3914 | 2026.08.17 23:52:52.918161 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_1_0 to 202608_6_6_0 into Compact with storage Full | ||
| 3915 | 2026.08.17 23:52:52.918651 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 38 | ||
| 3916 | 2026.08.17 23:52:52.920750 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 3917 | 2026.08.17 23:52:52.921115 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_1_0, total 8 rows starting from the beginning of the part | ||
| 3918 | 2026.08.17 23:52:52.923010 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_2_2_0, total 8 rows starting from the beginning of the part | ||
| 3919 | 2026.08.17 23:52:52.924854 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_3_3_0, total 8 rows starting from the beginning of the part | ||
| 3920 | 2026.08.17 23:52:52.926721 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_4_4_0, total 7 rows starting from the beginning of the part | ||
| 3921 | 2026.08.17 23:52:52.928542 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_5_5_0, total 8 rows starting from the beginning of the part | ||
| 3922 | 2026.08.17 23:52:52.930375 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_6_6_0, total 7 rows starting from the beginning of the part | ||
| 3923 | 2026.08.17 23:52:52.957323 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 46 rows, containing 679 columns (679 merged, 0 gathered) in 0.039224578 sec., 1172.734095443933 rows/sec., 2.12 MiB/sec. | ||
| 3924 | 2026.08.17 23:52:52.957915 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Trace> MergedBlockOutputStream: filled checksums 202608_1_6_1 (state Temporary) | ||
| 3925 | 2026.08.17 23:52:52.961338 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_6_1 to 202608_1_6_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 3926 | 2026.08.17 23:52:52.961366 [ 432207 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_6_1} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_1_0, 202608_6_6_0] -> 202608_1_6_1 | ||
| 3927 | 2026.08.17 23:52:52.961735 [ 432207 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.64 MiB. | ||
| 3928 | 2026.08.17 23:52:53.000306 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 427.76 MiB, peak 437.62 MiB, free memory in arenas 96.57 MiB, will set to 441.91 MiB (RSS), difference: 14.15 MiB | ||
| 3929 | 2026.08.17 23:52:53.030304 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Debug> executeQuery: Read 3 rows, 141.00 B in 4.670436 sec., 0.6423383170222224 rows/sec., 30.19 B/sec. | ||
| 3930 | 2026.08.17 23:52:53.030328 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Debug> MemoryTracker: Peak memory usage (for query): 4.15 MiB. | ||
| 3931 | 2026.08.17 23:52:53.030331 [ 433548 ] {680aa279-d059-461a-9f5e-60b39f1237ea} <Debug> TCPHandler: Processed in 4.670746451 sec. | ||
| 3932 | 2026.08.17 23:52:53.030608 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 3933 | 2026.08.17 23:52:53.030663 [ 433548 ] {e78704b5-17fe-4b81-beb6-c949f95dfb17} <Trace> executeQuery: Query span trace_id for opentelemetry log: 7e8fcb1f-01e3-6429-6d72-e02392538572 | ||
| 3934 | 2026.08.17 23:52:53.030735 [ 433548 ] {e78704b5-17fe-4b81-beb6-c949f95dfb17} <Debug> executeQuery: (from [::1]:56652) /* The version table contains metadata about the `oximeter` database */ | ||
| 3935 | CREATE TABLE IF NOT EXISTS oximeter.version ON CLUSTER oximeter_cluster ( value UInt64, timestamp DateTime64(9, 'UTC') ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) (stage: Complete) | ||
| 3936 | 2026.08.17 23:52:53.030775 [ 433548 ] {e78704b5-17fe-4b81-beb6-c949f95dfb17} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.version | ||
| 3937 | 2026.08.17 23:52:53.030820 [ 433548 ] {e78704b5-17fe-4b81-beb6-c949f95dfb17} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.version | ||
| 3938 | 2026.08.17 23:52:53.037756 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 3939 | 2026.08.17 23:52:53.037762 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 3940 | 2026.08.17 23:52:53.041043 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=22, queue_size=22, entries=query-0000000000..query-0000000021, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000020, last_skipped_entry_name=query-0000000019 | ||
| 3941 | 2026.08.17 23:52:53.041048 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000021 | ||
| 3942 | 2026.08.17 23:52:53.041050 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000021 | ||
| 3943 | 2026.08.17 23:52:53.041291 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000021 (CREATE TABLE IF NOT EXISTS oximeter.version UUID '4628b84b-456f-4f0e-9a48-6dc5825cfbd1' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp)) | ||
| 3944 | 2026.08.17 23:52:53.050512 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.version UUID '4628b84b-456f-4f0e-9a48-6dc5825cfbd1' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) | ||
| 3945 | 2026.08.17 23:52:53.050545 [ 432273 ] {30ccfeb6-6203-4911-a10c-305cc81c867e} <Trace> executeQuery: Query span trace_id for opentelemetry log: 7e8fcb1f-01e3-6429-6d72-e02392538572 | ||
| 3946 | 2026.08.17 23:52:53.050598 [ 432273 ] {30ccfeb6-6203-4911-a10c-305cc81c867e} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: e78704b5-17fe-4b81-beb6-c949f95dfb17) /* ddl_entry=query-0000000021 */ CREATE TABLE IF NOT EXISTS oximeter.version UUID '4628b84b-456f-4f0e-9a48-6dc5825cfbd1' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) (stage: Complete) | ||
| 3947 | 2026.08.17 23:52:53.050665 [ 432273 ] {30ccfeb6-6203-4911-a10c-305cc81c867e} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000021 */ CREATE TABLE IF NOT EXISTS oximeter.version UUID '4628b84b-456f-4f0e-9a48-6dc5825cfbd1' (`value` UInt64, `timestamp` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/version', '{replica}') ORDER BY (value, timestamp) | ||
| 3948 | 2026.08.17 23:52:53.059833 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 3949 | 2026.08.17 23:52:53.142057 [ 433548 ] {e78704b5-17fe-4b81-beb6-c949f95dfb17} <Debug> executeQuery: Read 3 rows, 141.00 B in 0.111384 sec., 26.933850463262228 rows/sec., 1.24 KiB/sec. | ||
| 3950 | 2026.08.17 23:52:53.142073 [ 433548 ] {e78704b5-17fe-4b81-beb6-c949f95dfb17} <Debug> TCPHandler: Processed in 0.11162482 sec. | ||
| 3951 | 2026.08.17 23:52:53.142185 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 3952 | 2026.08.17 23:52:53.142229 [ 433548 ] {bcd45197-4e8e-4d50-9280-ac5e06c5560f} <Trace> executeQuery: Query span trace_id for opentelemetry log: 0f27273a-965d-2ffb-461b-0595fbdcb9d1 | ||
| 3953 | 2026.08.17 23:52:53.142309 [ 433548 ] {bcd45197-4e8e-4d50-9280-ac5e06c5560f} <Debug> executeQuery: (from [::1]:56652) /* The measurement tables contain all individual samples from each timeseries. | ||
| 3954 | * | ||
| 3955 | * Each table stores a single datum type, and otherwise contains nearly the same | ||
| 3956 | * structure. The primary sorting key is on the timeseries name, key, and then | ||
| 3957 | * timestamp, so that all timeseries from the same schema are grouped, followed | ||
| 3958 | * by all samples from the same timeseries. | ||
| 3959 | * | ||
| 3960 | * This reflects that one usually looks up the _key_ in one or more field table, | ||
| 3961 | * and then uses that to index quickly into the measurements tables. | ||
| 3962 | */ | ||
| 3963 | CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local ON CLUSTER oximeter_cluster ( timeseries_name String, timeseries_key UInt64, start_time DateTime64(9, 'UTC'), timestamp DateTime64(9, 'UTC'), datum Nullable(Float64) ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + INTERVAL 30 DAY (stage: Complete) | ||
| 3964 | 2026.08.17 23:52:53.142335 [ 433548 ] {bcd45197-4e8e-4d50-9280-ac5e06c5560f} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.measurements_cumulativef64_local | ||
| 3965 | 2026.08.17 23:52:53.142384 [ 433548 ] {bcd45197-4e8e-4d50-9280-ac5e06c5560f} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.measurements_cumulativef64_local | ||
| 3966 | 2026.08.17 23:52:53.148843 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 3967 | 2026.08.17 23:52:53.148847 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 3968 | 2026.08.17 23:52:53.148934 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=23, queue_size=23, entries=query-0000000000..query-0000000022, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000021, last_skipped_entry_name=query-0000000019 | ||
| 3969 | 2026.08.17 23:52:53.148937 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000022 | ||
| 3970 | 2026.08.17 23:52:53.148940 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000022 | ||
| 3971 | 2026.08.17 23:52:53.152285 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000022 (CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID 'fa926c01-f931-4a9f-b027-b426f325ccc0' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30)) | ||
| 3972 | 2026.08.17 23:52:53.161436 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID 'fa926c01-f931-4a9f-b027-b426f325ccc0' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30) | ||
| 3973 | 2026.08.17 23:52:53.161462 [ 432273 ] {e826c8df-b6e4-4a95-9274-c476de1503aa} <Trace> executeQuery: Query span trace_id for opentelemetry log: 0f27273a-965d-2ffb-461b-0595fbdcb9d1 | ||
| 3974 | 2026.08.17 23:52:53.161517 [ 432273 ] {e826c8df-b6e4-4a95-9274-c476de1503aa} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: bcd45197-4e8e-4d50-9280-ac5e06c5560f) /* ddl_entry=query-0000000022 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID 'fa926c01-f931-4a9f-b027-b426f325ccc0' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30) (stage: Complete) | ||
| 3975 | 2026.08.17 23:52:53.161599 [ 432273 ] {e826c8df-b6e4-4a95-9274-c476de1503aa} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000022 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64_local UUID 'fa926c01-f931-4a9f-b027-b426f325ccc0' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/measurements_cumulativef64_local', '{replica}') ORDER BY (timeseries_name, timeseries_key, start_time, timestamp) TTL toDateTime(timestamp) + toIntervalDay(30) | ||
| 3976 | 2026.08.17 23:52:53.167553 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 3977 | 2026.08.17 23:52:53.262191 [ 433548 ] {bcd45197-4e8e-4d50-9280-ac5e06c5560f} <Debug> executeQuery: Read 3 rows, 141.00 B in 0.119952 sec., 25.010004001600638 rows/sec., 1.15 KiB/sec. | ||
| 3978 | 2026.08.17 23:52:53.262214 [ 433548 ] {bcd45197-4e8e-4d50-9280-ac5e06c5560f} <Debug> TCPHandler: Processed in 0.120072326 sec. | ||
| 3979 | 2026.08.17 23:52:53.262342 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 3980 | 2026.08.17 23:52:53.262383 [ 433548 ] {702b80e3-af8f-4743-808e-8ed1a53a4309} <Trace> executeQuery: Query span trace_id for opentelemetry log: d1822001-63fc-6eca-3501-3ec068599320 | ||
| 3981 | 2026.08.17 23:52:53.262427 [ 433548 ] {702b80e3-af8f-4743-808e-8ed1a53a4309} <Debug> executeQuery: (from [::1]:56652) CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 ON CLUSTER oximeter_cluster AS oximeter.measurements_cumulativef64_local ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 3982 | 2026.08.17 23:52:53.262444 [ 433548 ] {702b80e3-af8f-4743-808e-8ed1a53a4309} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.measurements_cumulativef64 | ||
| 3983 | 2026.08.17 23:52:53.262449 [ 433548 ] {702b80e3-af8f-4743-808e-8ed1a53a4309} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 3984 | 2026.08.17 23:52:53.262452 [ 433548 ] {702b80e3-af8f-4743-808e-8ed1a53a4309} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 3985 | 2026.08.17 23:52:53.262496 [ 433548 ] {702b80e3-af8f-4743-808e-8ed1a53a4309} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.measurements_cumulativef64 | ||
| 3986 | 2026.08.17 23:52:53.262498 [ 433548 ] {702b80e3-af8f-4743-808e-8ed1a53a4309} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 3987 | 2026.08.17 23:52:53.268804 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 3988 | 2026.08.17 23:52:53.268808 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 3989 | 2026.08.17 23:52:53.271956 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=24, queue_size=24, entries=query-0000000000..query-0000000023, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000022, last_skipped_entry_name=query-0000000019 | ||
| 3990 | 2026.08.17 23:52:53.271960 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000023 | ||
| 3991 | 2026.08.17 23:52:53.271962 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000023 | ||
| 3992 | 2026.08.17 23:52:53.272167 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000023 (CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '1c7e8baa-124b-488e-98ae-912d8d8035f2' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 3993 | 2026.08.17 23:52:53.275294 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '1c7e8baa-124b-488e-98ae-912d8d8035f2' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 3994 | 2026.08.17 23:52:53.275323 [ 432273 ] {7c1526e7-d937-4c6c-976d-46d49e8c3a9f} <Trace> executeQuery: Query span trace_id for opentelemetry log: d1822001-63fc-6eca-3501-3ec068599320 | ||
| 3995 | 2026.08.17 23:52:53.275380 [ 432273 ] {7c1526e7-d937-4c6c-976d-46d49e8c3a9f} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 702b80e3-af8f-4743-808e-8ed1a53a4309) /* ddl_entry=query-0000000023 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '1c7e8baa-124b-488e-98ae-912d8d8035f2' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 3996 | 2026.08.17 23:52:53.275455 [ 432273 ] {7c1526e7-d937-4c6c-976d-46d49e8c3a9f} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000023 */ CREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64 UUID '1c7e8baa-124b-488e-98ae-912d8d8035f2' (`timeseries_name` String, `timeseries_key` UInt64, `start_time` DateTime64(9, 'UTC'), `timestamp` DateTime64(9, 'UTC'), `datum` Nullable(Float64)) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'measurements_cumulativef64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 3997 | 2026.08.17 23:52:53.281312 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 3998 | 2026.08.17 23:52:53.375800 [ 433548 ] {702b80e3-af8f-4743-808e-8ed1a53a4309} <Debug> executeQuery: Read 3 rows, 141.00 B in 0.113408 sec., 26.453160270880364 rows/sec., 1.21 KiB/sec. | ||
| 3999 | 2026.08.17 23:52:53.375819 [ 433548 ] {702b80e3-af8f-4743-808e-8ed1a53a4309} <Debug> TCPHandler: Processed in 0.113521391 sec. | ||
| 4000 | 2026.08.17 23:52:53.375930 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4001 | 2026.08.17 23:52:53.375970 [ 433548 ] {73428e74-540c-4b15-a685-62b0888efb95} <Trace> executeQuery: Query span trace_id for opentelemetry log: 2c670ba3-f73a-1953-5062-9954240e4195 | ||
| 4002 | 2026.08.17 23:52:53.376057 [ 433548 ] {73428e74-540c-4b15-a685-62b0888efb95} <Debug> executeQuery: (from [::1]:56652) /* The field tables store named dimensions of each timeseries. | ||
| 4003 | * | ||
| 4004 | * As with the measurement tables, there is one field table for each field data | ||
| 4005 | * type. Fields are deduplicated by using the "replacing merge tree", though | ||
| 4006 | * this behavior **must not** be relied upon for query correctness. | ||
| 4007 | * | ||
| 4008 | * The index for the fields differs from the measurements, however. Rows are | ||
| 4009 | * sorted by timeseries name, then field name, field value, and finally | ||
| 4010 | * timeseries key. This reflects the most common pattern for looking them up: | ||
| 4011 | * by field name and possibly value, within a timeseries. The resulting keys are | ||
| 4012 | * usually then used to look up measurements. | ||
| 4013 | * | ||
| 4014 | * NOTE: We may want to consider a secondary index on these tables, sorting by | ||
| 4015 | * timeseries name and then key, since it would improve lookups where one | ||
| 4016 | * already has the key. Realistically though, these tables are quite small and | ||
| 4017 | * so performance benefits will be low in absolute terms. | ||
| 4018 | */ | ||
| 4019 | CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local ON CLUSTER oximeter_cluster ( timeseries_name String, timeseries_key UInt64, field_name String, field_value Int64, last_updated_at DateTime MATERIALIZED now() ) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + INTERVAL 30 DAY (stage: Complete) | ||
| 4020 | 2026.08.17 23:52:53.376078 [ 433548 ] {73428e74-540c-4b15-a685-62b0888efb95} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_i64_local | ||
| 4021 | 2026.08.17 23:52:53.376203 [ 433548 ] {73428e74-540c-4b15-a685-62b0888efb95} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_i64_local | ||
| 4022 | 2026.08.17 23:52:53.384245 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 4023 | 2026.08.17 23:52:53.384251 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 4024 | 2026.08.17 23:52:53.387316 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=25, queue_size=25, entries=query-0000000000..query-0000000024, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000023, last_skipped_entry_name=query-0000000019 | ||
| 4025 | 2026.08.17 23:52:53.387322 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000024 | ||
| 4026 | 2026.08.17 23:52:53.387325 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000024 | ||
| 4027 | 2026.08.17 23:52:53.387526 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000024 (CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '9a6deda6-8e53-450c-b73c-2d5dded3d2dd' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30)) | ||
| 4028 | 2026.08.17 23:52:53.396532 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '9a6deda6-8e53-450c-b73c-2d5dded3d2dd' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 4029 | 2026.08.17 23:52:53.396556 [ 432273 ] {55d51540-1f23-47e8-ba1a-ba9c144f78eb} <Trace> executeQuery: Query span trace_id for opentelemetry log: 2c670ba3-f73a-1953-5062-9954240e4195 | ||
| 4030 | 2026.08.17 23:52:53.396606 [ 432273 ] {55d51540-1f23-47e8-ba1a-ba9c144f78eb} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 73428e74-540c-4b15-a685-62b0888efb95) /* ddl_entry=query-0000000024 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '9a6deda6-8e53-450c-b73c-2d5dded3d2dd' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) (stage: Complete) | ||
| 4031 | 2026.08.17 23:52:53.396720 [ 432273 ] {55d51540-1f23-47e8-ba1a-ba9c144f78eb} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000024 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64_local UUID '9a6deda6-8e53-450c-b73c-2d5dded3d2dd' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_i64_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 4032 | 2026.08.17 23:52:53.405564 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 4033 | 2026.08.17 23:52:53.488080 [ 433548 ] {73428e74-540c-4b15-a685-62b0888efb95} <Debug> executeQuery: Read 3 rows, 141.00 B in 0.1121 sec., 26.761819803746654 rows/sec., 1.23 KiB/sec. | ||
| 4034 | 2026.08.17 23:52:53.488097 [ 433548 ] {73428e74-540c-4b15-a685-62b0888efb95} <Debug> TCPHandler: Processed in 0.112211231 sec. | ||
| 4035 | 2026.08.17 23:52:53.488199 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4036 | 2026.08.17 23:52:53.488238 [ 433548 ] {c32b927e-2f1f-410a-8847-47d75e3149a7} <Trace> executeQuery: Query span trace_id for opentelemetry log: ad335a2e-a58e-3149-a29c-0d2b17b9be8a | ||
| 4037 | 2026.08.17 23:52:53.488281 [ 433548 ] {c32b927e-2f1f-410a-8847-47d75e3149a7} <Debug> executeQuery: (from [::1]:56652) CREATE TABLE IF NOT EXISTS oximeter.fields_i64 ON CLUSTER oximeter_cluster AS oximeter.fields_i64_local ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 4038 | 2026.08.17 23:52:53.488297 [ 433548 ] {c32b927e-2f1f-410a-8847-47d75e3149a7} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_i64 | ||
| 4039 | 2026.08.17 23:52:53.488300 [ 433548 ] {c32b927e-2f1f-410a-8847-47d75e3149a7} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 4040 | 2026.08.17 23:52:53.488306 [ 433548 ] {c32b927e-2f1f-410a-8847-47d75e3149a7} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 4041 | 2026.08.17 23:52:53.488335 [ 433548 ] {c32b927e-2f1f-410a-8847-47d75e3149a7} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_i64 | ||
| 4042 | 2026.08.17 23:52:53.488337 [ 433548 ] {c32b927e-2f1f-410a-8847-47d75e3149a7} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 4043 | 2026.08.17 23:52:53.494575 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 4044 | 2026.08.17 23:52:53.494579 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 4045 | 2026.08.17 23:52:53.497662 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=26, queue_size=26, entries=query-0000000000..query-0000000025, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000024, last_skipped_entry_name=query-0000000019 | ||
| 4046 | 2026.08.17 23:52:53.497666 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000025 | ||
| 4047 | 2026.08.17 23:52:53.497668 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000025 | ||
| 4048 | 2026.08.17 23:52:53.500856 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000025 (CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID '6bae2d1b-42e7-4517-b0f8-c17720227858' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 4049 | 2026.08.17 23:52:53.503944 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID '6bae2d1b-42e7-4517-b0f8-c17720227858' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 4050 | 2026.08.17 23:52:53.503966 [ 432273 ] {799b2a71-8dc8-4dea-a79c-822f61c99dd0} <Trace> executeQuery: Query span trace_id for opentelemetry log: ad335a2e-a58e-3149-a29c-0d2b17b9be8a | ||
| 4051 | 2026.08.17 23:52:53.504003 [ 432273 ] {799b2a71-8dc8-4dea-a79c-822f61c99dd0} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: c32b927e-2f1f-410a-8847-47d75e3149a7) /* ddl_entry=query-0000000025 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID '6bae2d1b-42e7-4517-b0f8-c17720227858' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 4052 | 2026.08.17 23:52:53.504118 [ 432273 ] {799b2a71-8dc8-4dea-a79c-822f61c99dd0} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000025 */ CREATE TABLE IF NOT EXISTS oximeter.fields_i64 UUID '6bae2d1b-42e7-4517-b0f8-c17720227858' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` Int64, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_i64_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 4053 | 2026.08.17 23:52:53.509946 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 4054 | 2026.08.17 23:52:53.598480 [ 433548 ] {c32b927e-2f1f-410a-8847-47d75e3149a7} <Debug> executeQuery: Read 3 rows, 141.00 B in 0.110233 sec., 27.21508078343146 rows/sec., 1.25 KiB/sec. | ||
| 4055 | 2026.08.17 23:52:53.598496 [ 433548 ] {c32b927e-2f1f-410a-8847-47d75e3149a7} <Debug> TCPHandler: Processed in 0.110338839 sec. | ||
| 4056 | 2026.08.17 23:52:53.598579 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4057 | 2026.08.17 23:52:53.598618 [ 433548 ] {838e5c77-1260-4bc7-b509-ca0ae210d474} <Trace> executeQuery: Query span trace_id for opentelemetry log: 22f43aa1-0a08-fa74-dc3c-7116e51f8bee | ||
| 4058 | 2026.08.17 23:52:53.598691 [ 433548 ] {838e5c77-1260-4bc7-b509-ca0ae210d474} <Debug> executeQuery: (from [::1]:56652) CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local ON CLUSTER oximeter_cluster ( timeseries_name String, timeseries_key UInt64, field_name String, field_value UUID, last_updated_at DateTime MATERIALIZED now() ) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + INTERVAL 30 DAY (stage: Complete) | ||
| 4059 | 2026.08.17 23:52:53.598710 [ 433548 ] {838e5c77-1260-4bc7-b509-ca0ae210d474} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_uuid_local | ||
| 4060 | 2026.08.17 23:52:53.598787 [ 433548 ] {838e5c77-1260-4bc7-b509-ca0ae210d474} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_uuid_local | ||
| 4061 | 2026.08.17 23:52:53.606760 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 4062 | 2026.08.17 23:52:53.606764 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 4063 | 2026.08.17 23:52:53.610324 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=27, queue_size=27, entries=query-0000000000..query-0000000026, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000025, last_skipped_entry_name=query-0000000019 | ||
| 4064 | 2026.08.17 23:52:53.610327 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000026 | ||
| 4065 | 2026.08.17 23:52:53.610329 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000026 | ||
| 4066 | 2026.08.17 23:52:53.617072 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000026 (CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID '851f63af-5345-4349-8fa7-9dc67654fad8' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30)) | ||
| 4067 | 2026.08.17 23:52:53.623397 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID '851f63af-5345-4349-8fa7-9dc67654fad8' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 4068 | 2026.08.17 23:52:53.623420 [ 432273 ] {5493891a-3462-49ce-8c00-ecea558e6590} <Trace> executeQuery: Query span trace_id for opentelemetry log: 22f43aa1-0a08-fa74-dc3c-7116e51f8bee | ||
| 4069 | 2026.08.17 23:52:53.623470 [ 432273 ] {5493891a-3462-49ce-8c00-ecea558e6590} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 838e5c77-1260-4bc7-b509-ca0ae210d474) /* ddl_entry=query-0000000026 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID '851f63af-5345-4349-8fa7-9dc67654fad8' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) (stage: Complete) | ||
| 4070 | 2026.08.17 23:52:53.623578 [ 432273 ] {5493891a-3462-49ce-8c00-ecea558e6590} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000026 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid_local UUID '851f63af-5345-4349-8fa7-9dc67654fad8' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{shard}/fields_uuid_local', '{replica}') ORDER BY (timeseries_name, field_name, field_value, timeseries_key) TTL last_updated_at + toIntervalDay(30) | ||
| 4071 | 2026.08.17 23:52:53.629618 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 4072 | 2026.08.17 23:52:53.717769 [ 433548 ] {838e5c77-1260-4bc7-b509-ca0ae210d474} <Debug> executeQuery: Read 3 rows, 141.00 B in 0.119137 sec., 25.18109403459882 rows/sec., 1.16 KiB/sec. | ||
| 4073 | 2026.08.17 23:52:53.717786 [ 433548 ] {838e5c77-1260-4bc7-b509-ca0ae210d474} <Debug> TCPHandler: Processed in 0.119247035 sec. | ||
| 4074 | 2026.08.17 23:52:53.717896 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4075 | 2026.08.17 23:52:53.717935 [ 433548 ] {b0f670d9-bfd1-4760-9bba-c839e9238ec2} <Trace> executeQuery: Query span trace_id for opentelemetry log: 7d503ba1-8322-ec05-bacf-d12c558531a7 | ||
| 4076 | 2026.08.17 23:52:53.717977 [ 433548 ] {b0f670d9-bfd1-4760-9bba-c839e9238ec2} <Debug> executeQuery: (from [::1]:56652) CREATE TABLE IF NOT EXISTS oximeter.fields_uuid ON CLUSTER oximeter_cluster AS oximeter.fields_uuid_local ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 4077 | 2026.08.17 23:52:53.717992 [ 433548 ] {b0f670d9-bfd1-4760-9bba-c839e9238ec2} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_uuid | ||
| 4078 | 2026.08.17 23:52:53.717995 [ 433548 ] {b0f670d9-bfd1-4760-9bba-c839e9238ec2} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 4079 | 2026.08.17 23:52:53.717998 [ 433548 ] {b0f670d9-bfd1-4760-9bba-c839e9238ec2} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 4080 | 2026.08.17 23:52:53.718026 [ 433548 ] {b0f670d9-bfd1-4760-9bba-c839e9238ec2} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.fields_uuid | ||
| 4081 | 2026.08.17 23:52:53.718028 [ 433548 ] {b0f670d9-bfd1-4760-9bba-c839e9238ec2} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 4082 | 2026.08.17 23:52:53.724474 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 4083 | 2026.08.17 23:52:53.724478 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 4084 | 2026.08.17 23:52:53.724553 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=28, queue_size=28, entries=query-0000000000..query-0000000027, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000026, last_skipped_entry_name=query-0000000019 | ||
| 4085 | 2026.08.17 23:52:53.724558 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000027 | ||
| 4086 | 2026.08.17 23:52:53.724560 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000027 | ||
| 4087 | 2026.08.17 23:52:53.727819 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000027 (CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '406c7adf-1cf3-4de6-ab73-b7a16eca33c6' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 4088 | 2026.08.17 23:52:53.733966 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '406c7adf-1cf3-4de6-ab73-b7a16eca33c6' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 4089 | 2026.08.17 23:52:53.733989 [ 432273 ] {cdecc371-d440-4804-8c05-4a60cb524cb6} <Trace> executeQuery: Query span trace_id for opentelemetry log: 7d503ba1-8322-ec05-bacf-d12c558531a7 | ||
| 4090 | 2026.08.17 23:52:53.734028 [ 432273 ] {cdecc371-d440-4804-8c05-4a60cb524cb6} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: b0f670d9-bfd1-4760-9bba-c839e9238ec2) /* ddl_entry=query-0000000027 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '406c7adf-1cf3-4de6-ab73-b7a16eca33c6' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 4091 | 2026.08.17 23:52:53.734128 [ 432273 ] {cdecc371-d440-4804-8c05-4a60cb524cb6} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000027 */ CREATE TABLE IF NOT EXISTS oximeter.fields_uuid UUID '406c7adf-1cf3-4de6-ab73-b7a16eca33c6' (`timeseries_name` String, `timeseries_key` UInt64, `field_name` String, `field_value` UUID, `last_updated_at` DateTime MATERIALIZED now()) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'fields_uuid_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 4092 | 2026.08.17 23:52:53.739822 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 4093 | 2026.08.17 23:52:53.837600 [ 433548 ] {b0f670d9-bfd1-4760-9bba-c839e9238ec2} <Debug> executeQuery: Read 3 rows, 141.00 B in 0.119654 sec., 25.07229177461681 rows/sec., 1.15 KiB/sec. | ||
| 4094 | 2026.08.17 23:52:53.837618 [ 433548 ] {b0f670d9-bfd1-4760-9bba-c839e9238ec2} <Debug> TCPHandler: Processed in 0.119765466 sec. | ||
| 4095 | 2026.08.17 23:52:53.837745 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4096 | 2026.08.17 23:52:53.837794 [ 433548 ] {34824996-e41e-4330-bcca-adae0fd29b48} <Trace> executeQuery: Query span trace_id for opentelemetry log: dde3d52d-0cf7-409f-6bd1-f4e9077524ff | ||
| 4097 | 2026.08.17 23:52:53.837894 [ 433548 ] {34824996-e41e-4330-bcca-adae0fd29b48} <Debug> executeQuery: (from [::1]:56652) /* The timeseries schema table stores the extracted schema for the samples | ||
| 4098 | * oximeter collects. | ||
| 4099 | */ | ||
| 4100 | CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local ON CLUSTER oximeter_cluster ( timeseries_name String, fields Nested( name String, type Enum( 'Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13 ), source Enum( 'Target' = 1, 'Metric' = 2 ) ), datum_type Enum( 'Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27 ), created DateTime64(9, 'UTC') ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) (stage: Complete) | ||
| 4101 | 2026.08.17 23:52:53.837923 [ 433548 ] {34824996-e41e-4330-bcca-adae0fd29b48} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.timeseries_schema_local | ||
| 4102 | 2026.08.17 23:52:53.838041 [ 433548 ] {34824996-e41e-4330-bcca-adae0fd29b48} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.timeseries_schema_local | ||
| 4103 | 2026.08.17 23:52:53.844457 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 4104 | 2026.08.17 23:52:53.844463 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 4105 | 2026.08.17 23:52:53.847543 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=29, queue_size=29, entries=query-0000000000..query-0000000028, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000027, last_skipped_entry_name=query-0000000019 | ||
| 4106 | 2026.08.17 23:52:53.847548 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000028 | ||
| 4107 | 2026.08.17 23:52:53.847550 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000028 | ||
| 4108 | 2026.08.17 23:52:53.850886 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000028 (CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID 'c3db170a-857c-4375-9f45-d6439d6bb792' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name)) | ||
| 4109 | 2026.08.17 23:52:53.856915 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID 'c3db170a-857c-4375-9f45-d6439d6bb792' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) | ||
| 4110 | 2026.08.17 23:52:53.856947 [ 432273 ] {4de31bb7-0249-49ad-8b28-0190eb140d90} <Trace> executeQuery: Query span trace_id for opentelemetry log: dde3d52d-0cf7-409f-6bd1-f4e9077524ff | ||
| 4111 | 2026.08.17 23:52:53.857036 [ 432273 ] {4de31bb7-0249-49ad-8b28-0190eb140d90} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 34824996-e41e-4330-bcca-adae0fd29b48) /* ddl_entry=query-0000000028 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID 'c3db170a-857c-4375-9f45-d6439d6bb792' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) (stage: Complete) | ||
| 4112 | 2026.08.17 23:52:53.857229 [ 432273 ] {4de31bb7-0249-49ad-8b28-0190eb140d90} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000028 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema_local UUID 'c3db170a-857c-4375-9f45-d6439d6bb792' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/timeseries_schema_local', '{replica}') ORDER BY (timeseries_name, fields.name) | ||
| 4113 | 2026.08.17 23:52:53.865866 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 4114 | 2026.08.17 23:52:53.951489 [ 433548 ] {34824996-e41e-4330-bcca-adae0fd29b48} <Debug> executeQuery: Read 3 rows, 141.00 B in 0.113686 sec., 26.38847351476875 rows/sec., 1.21 KiB/sec. | ||
| 4115 | 2026.08.17 23:52:53.951509 [ 433548 ] {34824996-e41e-4330-bcca-adae0fd29b48} <Debug> TCPHandler: Processed in 0.113815441 sec. | ||
| 4116 | 2026.08.17 23:52:53.951712 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4117 | 2026.08.17 23:52:53.951766 [ 433548 ] {64bac3d9-8021-42f4-822c-1a8983c77ed7} <Trace> executeQuery: Query span trace_id for opentelemetry log: 62d9e2cd-6c78-4a27-e533-fe25a97e466b | ||
| 4118 | 2026.08.17 23:52:53.951815 [ 433548 ] {64bac3d9-8021-42f4-822c-1a8983c77ed7} <Debug> executeQuery: (from [::1]:56652) CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema ON CLUSTER oximeter_cluster AS oximeter.timeseries_schema_local ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 4119 | 2026.08.17 23:52:53.951836 [ 433548 ] {64bac3d9-8021-42f4-822c-1a8983c77ed7} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.timeseries_schema | ||
| 4120 | 2026.08.17 23:52:53.951841 [ 433548 ] {64bac3d9-8021-42f4-822c-1a8983c77ed7} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 4121 | 2026.08.17 23:52:53.951844 [ 433548 ] {64bac3d9-8021-42f4-822c-1a8983c77ed7} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 4122 | 2026.08.17 23:52:53.951940 [ 433548 ] {64bac3d9-8021-42f4-822c-1a8983c77ed7} <Trace> ContextAccess (default): Access granted: CREATE TABLE ON oximeter.timeseries_schema | ||
| 4123 | 2026.08.17 23:52:53.951942 [ 433548 ] {64bac3d9-8021-42f4-822c-1a8983c77ed7} <Trace> ContextAccess (default): Access granted: REMOTE ON *.* | ||
| 4124 | 2026.08.17 23:52:53.958213 [ 432273 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 4125 | 2026.08.17 23:52:53.958216 [ 432268 ] {} <Trace> DDLWorker: Too early to clean queue, will do it later. | ||
| 4126 | 2026.08.17 23:52:53.961306 [ 432273 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=30, queue_size=30, entries=query-0000000000..query-0000000029, first_failed_task_name=none, current_tasks_size=1, last_current_task=query-0000000028, last_skipped_entry_name=query-0000000019 | ||
| 4127 | 2026.08.17 23:52:53.961311 [ 432273 ] {} <Debug> DDLWorker: Will schedule 1 tasks starting from query-0000000029 | ||
| 4128 | 2026.08.17 23:52:53.961313 [ 432273 ] {} <Trace> DDLWorker: Checking task query-0000000029 | ||
| 4129 | 2026.08.17 23:52:53.961633 [ 432273 ] {} <Debug> DDLWorker: Processing task query-0000000029 (CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID 'fd0d2ebe-1312-4365-ba64-01e34ea76077' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1]))) | ||
| 4130 | 2026.08.17 23:52:53.967583 [ 432273 ] {} <Debug> DDLWorker: Executing query: CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID 'fd0d2ebe-1312-4365-ba64-01e34ea76077' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 4131 | 2026.08.17 23:52:53.967617 [ 432273 ] {693f8ace-b177-45fa-8be0-264937ae8842} <Trace> executeQuery: Query span trace_id for opentelemetry log: 62d9e2cd-6c78-4a27-e533-fe25a97e466b | ||
| 4132 | 2026.08.17 23:52:53.967690 [ 432273 ] {693f8ace-b177-45fa-8be0-264937ae8842} <Debug> executeQuery: (from 0.0.0.0:0, user: , initial_query_id: 64bac3d9-8021-42f4-822c-1a8983c77ed7) /* ddl_entry=query-0000000029 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID 'fd0d2ebe-1312-4365-ba64-01e34ea76077' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) (stage: Complete) | ||
| 4133 | 2026.08.17 23:52:53.967818 [ 432273 ] {693f8ace-b177-45fa-8be0-264937ae8842} <Debug> DDLWorker: Executed query: /* ddl_entry=query-0000000029 */ CREATE TABLE IF NOT EXISTS oximeter.timeseries_schema UUID 'fd0d2ebe-1312-4365-ba64-01e34ea76077' (`timeseries_name` String, `fields.name` Array(String), `fields.type` Array(Enum8('Bool' = 1, 'I64' = 2, 'IpAddr' = 3, 'String' = 4, 'Uuid' = 6, 'I8' = 7, 'U8' = 8, 'I16' = 9, 'U16' = 10, 'I32' = 11, 'U32' = 12, 'U64' = 13)), `fields.source` Array(Enum8('Target' = 1, 'Metric' = 2)), `datum_type` Enum8('Bool' = 1, 'I64' = 2, 'F64' = 3, 'String' = 4, 'Bytes' = 5, 'CumulativeI64' = 6, 'CumulativeF64' = 7, 'HistogramI64' = 8, 'HistogramF64' = 9, 'I8' = 10, 'U8' = 11, 'I16' = 12, 'U16' = 13, 'I32' = 14, 'U32' = 15, 'U64' = 16, 'F32' = 17, 'CumulativeU64' = 18, 'CumulativeF32' = 19, 'HistogramI8' = 20, 'HistogramU8' = 21, 'HistogramI16' = 22, 'HistogramU16' = 23, 'HistogramI32' = 24, 'HistogramU32' = 25, 'HistogramU64' = 26, 'HistogramF32' = 27), `created` DateTime64(9, 'UTC')) ENGINE = Distributed('oximeter_cluster', 'oximeter', 'timeseries_schema_local', xxHash64(splitByChar(':', timeseries_name)[1])) | ||
| 4134 | 2026.08.17 23:52:53.973606 [ 432273 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 4135 | 2026.08.17 23:52:54.062325 [ 433548 ] {64bac3d9-8021-42f4-822c-1a8983c77ed7} <Debug> executeQuery: Read 3 rows, 141.00 B in 0.110549 sec., 27.137287537652988 rows/sec., 1.25 KiB/sec. | ||
| 4136 | 2026.08.17 23:52:54.062346 [ 433548 ] {64bac3d9-8021-42f4-822c-1a8983c77ed7} <Debug> TCPHandler: Processed in 0.110787389 sec. | ||
| 4137 | 2026.08.17 23:52:54.062662 [ 431938 ] {} <Trace> TCP-Session: 609225df-4b76-40ae-9458-3310c59c57d9 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4138 | 2026.08.17 23:52:54.062727 [ 431938 ] {8c889be2-3131-4666-87fb-1201daa18e26} <Trace> executeQuery: Query span trace_id for opentelemetry log: b2efdcf5-80d5-b9b3-83e1-5b619dfe7b5c | ||
| 4139 | 2026.08.17 23:52:54.062782 [ 431938 ] {8c889be2-3131-4666-87fb-1201daa18e26} <Debug> executeQuery: (from [::1]:56654) SELECT name FROM system.tables WHERE database = 'oximeter' AND engine = 'ReplicatedMergeTree' (stage: Complete) | ||
| 4140 | 2026.08.17 23:52:54.063022 [ 431938 ] {8c889be2-3131-4666-87fb-1201daa18e26} <Trace> ContextAccess (default): Access granted: SELECT(database, name, engine) ON system.tables | ||
| 4141 | 2026.08.17 23:52:54.063122 [ 431938 ] {8c889be2-3131-4666-87fb-1201daa18e26} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 4142 | 2026.08.17 23:52:54.063306 [ 432294 ] {8c889be2-3131-4666-87fb-1201daa18e26} <Trace> ContextAccess (default): Access granted: SHOW TABLES ON *.* | ||
| 4143 | 2026.08.17 23:52:54.063422 [ 431938 ] {8c889be2-3131-4666-87fb-1201daa18e26} <Debug> executeQuery: Read 9 rows, 631.00 B in 0.000687 sec., 13100.436681222707 rows/sec., 896.96 KiB/sec. | ||
| 4144 | 2026.08.17 23:52:54.063441 [ 431938 ] {8c889be2-3131-4666-87fb-1201daa18e26} <Debug> TCPHandler: Processed in 0.00083406 sec. | ||
| 4145 | 2026.08.17 23:52:54.066717 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4146 | 2026.08.17 23:52:54.066763 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> executeQuery: Query span trace_id for opentelemetry log: ab161466-cbab-91e5-3dce-4490bd870cc4 | ||
| 4147 | 2026.08.17 23:52:54.066804 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Debug> executeQuery: (from [::1]:56652) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 4148 | 2026.08.17 23:52:54.066997 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 4149 | 2026.08.17 23:52:54.067116 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 4150 | 2026.08.17 23:52:54.067246 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 4151 | 2026.08.17 23:52:54.067254 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name IN ('virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 4152 | 2026.08.17 23:52:54.067322 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 4153 | 2026.08.17 23:52:54.067347 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 4154 | 2026.08.17 23:52:54.067376 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 4155 | 2026.08.17 23:52:54.067443 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Key condition: (column 0 in 1-element set) | ||
| 4156 | 2026.08.17 23:52:54.067454 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 4157 | 2026.08.17 23:52:54.067465 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4158 | 2026.08.17 23:52:54.067468 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4159 | 2026.08.17 23:52:54.067476 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found continuous range in 2 steps | ||
| 4160 | 2026.08.17 23:52:54.067488 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 4161 | 2026.08.17 23:52:54.067492 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 4162 | 2026.08.17 23:52:54.067511 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "in(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4163 | 2026.08.17 23:52:54.067516 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 1 rows starting from 0 | ||
| 4164 | 2026.08.17 23:52:54.068050 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.001277 sec., 783.0853563038372 rows/sec., 96.36 KiB/sec. | ||
| 4165 | 2026.08.17 23:52:54.068106 [ 433548 ] {09f930b0-f33a-45a6-b4c5-c6981bec49f4} <Debug> TCPHandler: Processed in 0.001436031 sec. | ||
| 4166 | 2026.08.17 23:52:54.074981 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4167 | 2026.08.17 23:52:54.075021 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Trace> executeQuery: Query span trace_id for opentelemetry log: 46957fb0-770b-ad1d-9b72-6cc9bbd215f7 | ||
| 4168 | 2026.08.17 23:52:54.075052 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Debug> executeQuery: (from [::1]:56652) INSERT INTO oximeter.fields_i64 FORMAT Native (stage: Complete) | ||
| 4169 | 2026.08.17 23:52:54.075073 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 4170 | 2026.08.17 23:52:54.075409 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64_local | ||
| 4171 | 2026.08.17 23:52:54.075570 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4172 | 2026.08.17 23:52:54.075582 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 4173 | 2026.08.17 23:52:54.075780 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Trace> MergedBlockOutputStream: filled checksums all_1_1_0 (state Temporary) | ||
| 4174 | 2026.08.17 23:52:54.075879 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Replicated OutputStream): Wrote block with ID 'all_2147739120618179471_17478915608710332536', 8 rows | ||
| 4175 | 2026.08.17 23:52:54.079583 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Renaming temporary part tmp_insert_all_1_1_0 to all_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4176 | 2026.08.17 23:52:54.082903 [ 432079 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73437095936, merge_with_ttl_allowedtrue) | ||
| 4177 | 2026.08.17 23:52:54.082926 [ 432079 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 4178 | 2026.08.17 23:52:54.082949 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Debug> executeQuery: Read 8 rows, 512.00 B in 0.00792 sec., 1010.1010101010102 rows/sec., 63.13 KiB/sec. | ||
| 4179 | 2026.08.17 23:52:54.082964 [ 433548 ] {b4ec02fd-a7a7-4be7-bd25-e642aa579dd6} <Debug> TCPHandler: Processed in 0.008024646 sec. | ||
| 4180 | 2026.08.17 23:52:54.083011 [ 432071 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000004 - log-0000000004 | ||
| 4181 | 2026.08.17 23:52:54.083116 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4182 | 2026.08.17 23:52:54.083167 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Trace> executeQuery: Query span trace_id for opentelemetry log: 7fe3f53b-0228-10f7-7de8-284438abb366 | ||
| 4183 | 2026.08.17 23:52:54.083188 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Debug> executeQuery: (from [::1]:56652) INSERT INTO oximeter.fields_uuid FORMAT Native (stage: Complete) | ||
| 4184 | 2026.08.17 23:52:54.083206 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 4185 | 2026.08.17 23:52:54.083486 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid_local | ||
| 4186 | 2026.08.17 23:52:54.083621 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4187 | 2026.08.17 23:52:54.083631 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 4188 | 2026.08.17 23:52:54.083783 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Trace> MergedBlockOutputStream: filled checksums all_1_1_0 (state Temporary) | ||
| 4189 | 2026.08.17 23:52:54.083865 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Replicated OutputStream): Wrote block with ID 'all_1633397349257524503_339141090191839785', 16 rows | ||
| 4190 | 2026.08.17 23:52:54.086468 [ 432071 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000004 to queue with type GET_PART with virtual parts [all_4_4_0] | ||
| 4191 | 2026.08.17 23:52:54.086475 [ 432071 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 4192 | 2026.08.17 23:52:54.089578 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Renaming temporary part tmp_insert_all_1_1_0 to all_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4193 | 2026.08.17 23:52:54.095869 [ 432075 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73437052928, merge_with_ttl_allowedtrue) | ||
| 4194 | 2026.08.17 23:52:54.095878 [ 432075 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 4195 | 2026.08.17 23:52:54.095933 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Debug> executeQuery: Read 16 rows, 1.20 KiB in 0.012757 sec., 1254.2133730500902 rows/sec., 93.70 KiB/sec. | ||
| 4196 | 2026.08.17 23:52:54.095948 [ 433548 ] {236610d6-ae9e-4521-8156-1ffc79668b7c} <Debug> TCPHandler: Processed in 0.012872859 sec. | ||
| 4197 | 2026.08.17 23:52:54.096069 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4198 | 2026.08.17 23:52:54.096108 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Trace> executeQuery: Query span trace_id for opentelemetry log: 322e54b4-e496-1835-7ab9-ceac6ebbe18f | ||
| 4199 | 2026.08.17 23:52:54.096130 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Debug> executeQuery: (from [::1]:56652) INSERT INTO oximeter.measurements_cumulativef64 FORMAT Native (stage: Complete) | ||
| 4200 | 2026.08.17 23:52:54.096155 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 4201 | 2026.08.17 23:52:54.096527 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Trace> ContextAccess (default): Access granted: INSERT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64_local | ||
| 4202 | 2026.08.17 23:52:54.096642 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4203 | 2026.08.17 23:52:54.096651 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 4204 | 2026.08.17 23:52:54.098207 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Trace> MergedBlockOutputStream: filled checksums all_1_1_0 (state Temporary) | ||
| 4205 | 2026.08.17 23:52:54.098312 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Replicated OutputStream): Wrote block with ID 'all_17957856226428163130_8701820713366769330', 400 rows | ||
| 4206 | 2026.08.17 23:52:54.105252 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Renaming temporary part tmp_insert_all_1_1_0 to all_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4207 | 2026.08.17 23:52:54.105305 [ 432044 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000004 - log-0000000004 | ||
| 4208 | 2026.08.17 23:52:54.108464 [ 432107 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73437007872, merge_with_ttl_allowedtrue) | ||
| 4209 | 2026.08.17 23:52:54.108471 [ 432107 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 4210 | 2026.08.17 23:52:54.108518 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Debug> executeQuery: Read 400 rows, 25.78 KiB in 0.012403 sec., 32250.26203337902 rows/sec., 2.03 MiB/sec. | ||
| 4211 | 2026.08.17 23:52:54.109060 [ 431939 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47254, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 4212 | 2026.08.17 23:52:54.109066 [ 433548 ] {8e3a8109-e26e-429c-88a6-f027d582cbfb} <Debug> TCPHandler: Processed in 0.01303969 sec. | ||
| 4213 | 2026.08.17 23:52:54.109077 [ 434092 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47256, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 4214 | 2026.08.17 23:52:54.109100 [ 431939 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_i64_local%2Freplicas%2F1&part=all_4_4_0&client_protocol_version=8&compress=false | ||
| 4215 | 2026.08.17 23:52:54.109107 [ 431939 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Replicated PartsService): Sending part all_4_4_0 | ||
| 4216 | 2026.08.17 23:52:54.109119 [ 434092 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_i64_local%2Freplicas%2F1&part=all_4_4_0&client_protocol_version=8&compress=false | ||
| 4217 | 2026.08.17 23:52:54.109125 [ 434092 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (Replicated PartsService): Sending part all_4_4_0 | ||
| 4218 | 2026.08.17 23:52:54.109305 [ 431939 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 4219 | 2026.08.17 23:52:54.109308 [ 434092 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 4220 | 2026.08.17 23:52:54.109357 [ 431938 ] {} <Trace> TCP-Session: 609225df-4b76-40ae-9458-3310c59c57d9 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4221 | 2026.08.17 23:52:54.109404 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> executeQuery: Query span trace_id for opentelemetry log: 481c5a48-83ec-df32-30fc-bbb0ce9b9448 | ||
| 4222 | 2026.08.17 23:52:54.109444 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Debug> executeQuery: (from [::1]:56654) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 4223 | 2026.08.17 23:52:54.109597 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 4224 | 2026.08.17 23:52:54.111196 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 4225 | 2026.08.17 23:52:54.111287 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 4226 | 2026.08.17 23:52:54.111293 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name IN ('virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 4227 | 2026.08.17 23:52:54.111348 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 4228 | 2026.08.17 23:52:54.111367 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 4229 | 2026.08.17 23:52:54.111393 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 4230 | 2026.08.17 23:52:54.111437 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Key condition: (column 0 in 1-element set) | ||
| 4231 | 2026.08.17 23:52:54.111442 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 4232 | 2026.08.17 23:52:54.111448 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4233 | 2026.08.17 23:52:54.111452 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4234 | 2026.08.17 23:52:54.111455 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found continuous range in 2 steps | ||
| 4235 | 2026.08.17 23:52:54.111461 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 4236 | 2026.08.17 23:52:54.111465 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 4237 | 2026.08.17 23:52:54.111477 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "in(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4238 | 2026.08.17 23:52:54.111480 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 1 rows starting from 0 | ||
| 4239 | 2026.08.17 23:52:54.111857 [ 432044 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000004 to queue with type GET_PART with virtual parts [all_4_4_0] | ||
| 4240 | 2026.08.17 23:52:54.111865 [ 432044 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 4241 | 2026.08.17 23:52:54.111939 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.002524 sec., 396.19651347068145 rows/sec., 48.75 KiB/sec. | ||
| 4242 | 2026.08.17 23:52:54.111996 [ 431938 ] {586eab84-44a0-46ef-8f22-87a87109262b} <Debug> TCPHandler: Processed in 0.002685112 sec. | ||
| 4243 | 2026.08.17 23:52:54.112464 [ 431938 ] {} <Trace> TCP-Session: 609225df-4b76-40ae-9458-3310c59c57d9 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4244 | 2026.08.17 23:52:54.112503 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> executeQuery: Query span trace_id for opentelemetry log: 920c97f9-e602-4e3c-a77f-94778cd793ba | ||
| 4245 | 2026.08.17 23:52:54.112540 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Debug> executeQuery: (from [::1]:56654) SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name IN ('virtual_machine:cpu_busy') FORMAT Native (stage: Complete) | ||
| 4246 | 2026.08.17 23:52:54.112678 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 4247 | 2026.08.17 23:52:54.112764 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema | ||
| 4248 | 2026.08.17 23:52:54.112839 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 4249 | 2026.08.17 23:52:54.112844 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name IN ('virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 4250 | 2026.08.17 23:52:54.112894 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, `fields.name`, `fields.type`, `fields.source`, datum_type, created) ON oximeter.timeseries_schema_local | ||
| 4251 | 2026.08.17 23:52:54.112910 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 4252 | 2026.08.17 23:52:54.112933 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 4253 | 2026.08.17 23:52:54.114490 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Key condition: (column 0 in 1-element set) | ||
| 4254 | 2026.08.17 23:52:54.114496 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 4255 | 2026.08.17 23:52:54.114501 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4256 | 2026.08.17 23:52:54.114504 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4257 | 2026.08.17 23:52:54.114507 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Found continuous range in 2 steps | ||
| 4258 | 2026.08.17 23:52:54.114513 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Debug> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1/1 marks by primary key, 1 marks to read from 1 ranges | ||
| 4259 | 2026.08.17 23:52:54.114520 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 4260 | 2026.08.17 23:52:54.114539 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "in(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4261 | 2026.08.17 23:52:54.114543 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 1 rows starting from 0 | ||
| 4262 | 2026.08.17 23:52:54.114973 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Debug> executeQuery: Read 1 rows, 126.00 B in 0.002461 sec., 406.33888663145063 rows/sec., 50.00 KiB/sec. | ||
| 4263 | 2026.08.17 23:52:54.115029 [ 431938 ] {185c1be9-f0d9-4aba-a6bd-ba8a3f6119ed} <Debug> TCPHandler: Processed in 0.002605502 sec. | ||
| 4264 | 2026.08.17 23:52:54.115283 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4265 | 2026.08.17 23:52:54.115307 [ 432050 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000003 - log-0000000003 | ||
| 4266 | 2026.08.17 23:52:54.115329 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> executeQuery: Query span trace_id for opentelemetry log: b321e016-3f75-8be7-4ca5-ab6faa0754a3 | ||
| 4267 | 2026.08.17 23:52:54.115446 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> executeQuery: (from [::1]:56652) SELECT fields_i64_pivot.cpu_id AS cpu_id, fields_uuid_pivot.instance_id AS instance_id, fields_uuid_pivot.project_id AS project_id, fields_uuid_pivot.timeseries_key AS timeseries_key FROM ( SELECT timeseries_key, anyIf(field_value, field_name = 'instance_id') AS instance_id, anyIf(field_value, field_name = 'project_id') AS project_id FROM oximeter.fields_uuid WHERE timeseries_name = 'virtual_machine:cpu_busy' GROUP BY timeseries_key ) AS fields_uuid_pivot JOIN ( SELECT timeseries_key, anyIf(field_value, field_name = 'cpu_id') AS cpu_id FROM oximeter.fields_i64 WHERE timeseries_name = 'virtual_machine:cpu_busy' GROUP BY timeseries_key ) AS fields_i64_pivot ON fields_uuid_pivot.timeseries_key = fields_i64_pivot.timeseries_key (stage: Complete) | ||
| 4268 | 2026.08.17 23:52:54.115692 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 4269 | 2026.08.17 23:52:54.115804 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 4270 | 2026.08.17 23:52:54.115919 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 4271 | 2026.08.17 23:52:54.116167 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 4272 | 2026.08.17 23:52:54.116251 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64 | ||
| 4273 | 2026.08.17 23:52:54.117488 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 4274 | 2026.08.17 23:52:54.117494 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name = 'virtual_machine:cpu_busy'" moved to PREWHERE | ||
| 4275 | 2026.08.17 23:52:54.117548 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_i64_local | ||
| 4276 | 2026.08.17 23:52:54.117568 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 4277 | 2026.08.17 23:52:54.117594 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 4278 | 2026.08.17 23:52:54.117628 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> HashJoin: (0x7feb314fc518) Datatype: EMPTY, kind: Inner, strictness: All, right header: fields_i64_pivot.timeseries_key UInt64 UInt64(size = 0), cpu_id Int64 Int64(size = 0) | ||
| 4279 | 2026.08.17 23:52:54.117632 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> HashJoin: (0x7feb314fc518) Keys: [(timeseries_key) = (fields_i64_pivot.timeseries_key)] | ||
| 4280 | 2026.08.17 23:52:54.117809 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 4281 | 2026.08.17 23:52:54.117929 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid | ||
| 4282 | 2026.08.17 23:52:54.118029 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 0 | ||
| 4283 | 2026.08.17 23:52:54.118035 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "timeseries_name = 'virtual_machine:cpu_busy'" moved to PREWHERE | ||
| 4284 | 2026.08.17 23:52:54.118104 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, field_name, field_value) ON oximeter.fields_uuid_local | ||
| 4285 | 2026.08.17 23:52:54.118120 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 4286 | 2026.08.17 23:52:54.118144 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 4287 | 2026.08.17 23:52:54.118146 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 4288 | 2026.08.17 23:52:54.118154 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> JoiningTransform: Before join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0)' | ||
| 4289 | 2026.08.17 23:52:54.118167 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> JoiningTransform: After join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0), cpu_id Int64 Int64(size = 0)' | ||
| 4290 | 2026.08.17 23:52:54.118429 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Key condition: (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']) | ||
| 4291 | 2026.08.17 23:52:54.118499 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 4292 | 2026.08.17 23:52:54.118500 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_1_1_0 (2 marks) | ||
| 4293 | 2026.08.17 23:52:54.118506 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_3_3_0 (2 marks) | ||
| 4294 | 2026.08.17 23:52:54.118509 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_4_4_0 (2 marks) | ||
| 4295 | 2026.08.17 23:52:54.118515 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4296 | 2026.08.17 23:52:54.118521 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4297 | 2026.08.17 23:52:54.118515 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4298 | 2026.08.17 23:52:54.118533 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4299 | 2026.08.17 23:52:54.118537 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 4300 | 2026.08.17 23:52:54.118517 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4301 | 2026.08.17 23:52:54.118526 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4302 | 2026.08.17 23:52:54.118548 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4303 | 2026.08.17 23:52:54.118526 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4304 | 2026.08.17 23:52:54.118516 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Running binary search on index range for part all_2_2_0 (2 marks) | ||
| 4305 | 2026.08.17 23:52:54.118557 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 4306 | 2026.08.17 23:52:54.118551 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 4307 | 2026.08.17 23:52:54.118551 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 4308 | 2026.08.17 23:52:54.118564 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4309 | 2026.08.17 23:52:54.118570 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4310 | 2026.08.17 23:52:54.118574 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Found continuous range in 2 steps | ||
| 4311 | 2026.08.17 23:52:54.118614 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Selected 5/5 parts by partition key, 5 parts by primary key, 5/5 marks by primary key, 5 marks to read from 5 ranges | ||
| 4312 | 2026.08.17 23:52:54.118621 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 4313 | 2026.08.17 23:52:54.118673 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> MergeTreeReadPool: min_marks_for_concurrent_read=24 | ||
| 4314 | 2026.08.17 23:52:54.118681 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (SelectExecutor): Reading approx. 80 rows with 5 streams | ||
| 4315 | 2026.08.17 23:52:54.118691 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4316 | 2026.08.17 23:52:54.118703 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4317 | 2026.08.17 23:52:54.118709 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4318 | 2026.08.17 23:52:54.118715 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4319 | 2026.08.17 23:52:54.118721 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4320 | 2026.08.17 23:52:54.118855 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Key condition: (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']) | ||
| 4321 | 2026.08.17 23:52:54.118902 [ 432294 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_4_4_0 (2 marks) | ||
| 4322 | 2026.08.17 23:52:54.118904 [ 432293 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_1_1_0 (2 marks) | ||
| 4323 | 2026.08.17 23:52:54.118905 [ 432292 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_3_3_0 (2 marks) | ||
| 4324 | 2026.08.17 23:52:54.118906 [ 432271 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_2_2_0 (2 marks) | ||
| 4325 | 2026.08.17 23:52:54.118913 [ 432292 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4326 | 2026.08.17 23:52:54.118916 [ 432292 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4327 | 2026.08.17 23:52:54.118917 [ 432293 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4328 | 2026.08.17 23:52:54.118921 [ 432271 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4329 | 2026.08.17 23:52:54.118904 [ 432286 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Running binary search on index range for part all_0_0_0 (2 marks) | ||
| 4330 | 2026.08.17 23:52:54.118927 [ 432271 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4331 | 2026.08.17 23:52:54.118921 [ 432292 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 4332 | 2026.08.17 23:52:54.118931 [ 432271 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 4333 | 2026.08.17 23:52:54.118933 [ 432286 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4334 | 2026.08.17 23:52:54.118922 [ 432293 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4335 | 2026.08.17 23:52:54.118907 [ 432294 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (LEFT) boundary mark: 0 | ||
| 4336 | 2026.08.17 23:52:54.118944 [ 432293 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 4337 | 2026.08.17 23:52:54.118945 [ 432294 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4338 | 2026.08.17 23:52:54.118938 [ 432286 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found (RIGHT) boundary mark: 2 | ||
| 4339 | 2026.08.17 23:52:54.118950 [ 432294 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 4340 | 2026.08.17 23:52:54.118953 [ 432286 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Found continuous range in 2 steps | ||
| 4341 | 2026.08.17 23:52:54.118984 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Selected 5/5 parts by partition key, 5 parts by primary key, 5/5 marks by primary key, 5 marks to read from 5 ranges | ||
| 4342 | 2026.08.17 23:52:54.118988 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Spreading mark ranges among streams (default reading) | ||
| 4343 | 2026.08.17 23:52:54.119023 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> MergeTreeReadPool: min_marks_for_concurrent_read=24 | ||
| 4344 | 2026.08.17 23:52:54.119027 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (SelectExecutor): Reading approx. 40 rows with 5 streams | ||
| 4345 | 2026.08.17 23:52:54.119034 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4346 | 2026.08.17 23:52:54.119042 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4347 | 2026.08.17 23:52:54.119048 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4348 | 2026.08.17 23:52:54.119054 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4349 | 2026.08.17 23:52:54.119062 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 1 steps: "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4350 | 2026.08.17 23:52:54.119168 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> JoiningTransform: Before join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0)' | ||
| 4351 | 2026.08.17 23:52:54.119176 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> JoiningTransform: After join block: 'timeseries_key UInt64 UInt64(size = 0), instance_id UUID UUID(size = 0), project_id UUID UUID(size = 0), cpu_id Int64 Int64(size = 0)' | ||
| 4352 | 2026.08.17 23:52:54.119592 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4353 | 2026.08.17 23:52:54.120560 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=1559201803930241434 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4354 | 2026.08.17 23:52:54.119609 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4355 | 2026.08.17 23:52:54.119756 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4356 | 2026.08.17 23:52:54.119792 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4357 | 2026.08.17 23:52:54.119606 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4358 | 2026.08.17 23:52:54.120567 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4359 | 2026.08.17 23:52:54.120578 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=1559201803930241434 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4360 | 2026.08.17 23:52:54.120627 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4361 | 2026.08.17 23:52:54.120629 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=1559201803930241434 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4362 | 2026.08.17 23:52:54.120634 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4363 | 2026.08.17 23:52:54.120636 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=1559201803930241434 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4364 | 2026.08.17 23:52:54.120642 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4365 | 2026.08.17 23:52:54.120647 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=1559201803930241434 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4366 | 2026.08.17 23:52:54.120651 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.001542681 sec. (5185.777 rows/sec., 86.09 KiB/sec.) | ||
| 4367 | 2026.08.17 23:52:54.120652 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4368 | 2026.08.17 23:52:54.120653 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.001553701 sec. (5148.996 rows/sec., 85.48 KiB/sec.) | ||
| 4369 | 2026.08.17 23:52:54.120653 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.001547571 sec. (5169.391 rows/sec., 85.82 KiB/sec.) | ||
| 4370 | 2026.08.17 23:52:54.120662 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.001557141 sec. (5137.621 rows/sec., 85.29 KiB/sec.) | ||
| 4371 | 2026.08.17 23:52:54.120653 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 8 to 8 rows (from 136.00 B) in 0.001551371 sec. (5156.729 rows/sec., 85.61 KiB/sec.) | ||
| 4372 | 2026.08.17 23:52:54.120671 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Merging aggregated data | ||
| 4373 | 2026.08.17 23:52:54.120675 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Statistics updated for key=1559201803930241434: new sum_of_sizes=40, median_size=8 | ||
| 4374 | 2026.08.17 23:52:54.127626 [ 432050 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000003 to queue with type GET_PART with virtual parts [all_4_4_0] | ||
| 4375 | 2026.08.17 23:52:54.127633 [ 432050 ] {} <Debug> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 4376 | 2026.08.17 23:52:54.139781 [ 431939 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47254, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 4377 | 2026.08.17 23:52:54.139796 [ 431939 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_uuid_local%2Freplicas%2F1&part=all_4_4_0&client_protocol_version=8&compress=false | ||
| 4378 | 2026.08.17 23:52:54.139800 [ 431939 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Replicated PartsService): Sending part all_4_4_0 | ||
| 4379 | 2026.08.17 23:52:54.148858 [ 434092 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47256, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 4380 | 2026.08.17 23:52:54.148875 [ 434092 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Ffields_uuid_local%2Freplicas%2F1&part=all_4_4_0&client_protocol_version=8&compress=false | ||
| 4381 | 2026.08.17 23:52:54.148880 [ 434092 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (Replicated PartsService): Sending part all_4_4_0 | ||
| 4382 | 2026.08.17 23:52:54.155168 [ 432022 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000005 - log-0000000005 | ||
| 4383 | 2026.08.17 23:52:54.159208 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4384 | 2026.08.17 23:52:54.159214 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=11757313445726935263 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4385 | 2026.08.17 23:52:54.159217 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4386 | 2026.08.17 23:52:54.159228 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4387 | 2026.08.17 23:52:54.159235 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=11757313445726935263 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4388 | 2026.08.17 23:52:54.159244 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4389 | 2026.08.17 23:52:54.159246 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4390 | 2026.08.17 23:52:54.159249 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4391 | 2026.08.17 23:52:54.159264 [ 432264 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.040482759 sec. (395.230 rows/sec., 10.04 KiB/sec.) | ||
| 4392 | 2026.08.17 23:52:54.159266 [ 432259 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.040483229 sec. (395.225 rows/sec., 10.04 KiB/sec.) | ||
| 4393 | 2026.08.17 23:52:54.159269 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregating | ||
| 4394 | 2026.08.17 23:52:54.159273 [ 431939 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 4395 | 2026.08.17 23:52:54.159278 [ 434092 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 4396 | 2026.08.17 23:52:54.159255 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=11757313445726935263 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4397 | 2026.08.17 23:52:54.159287 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4398 | 2026.08.17 23:52:54.159291 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=11757313445726935263 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4399 | 2026.08.17 23:52:54.159295 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4400 | 2026.08.17 23:52:54.159300 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: An entry for key=11757313445726935263 found in cache: sum_of_sizes=16, median_size=8 | ||
| 4401 | 2026.08.17 23:52:54.159305 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Aggregation method: key64 | ||
| 4402 | 2026.08.17 23:52:54.159312 [ 432265 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.040536669 sec. (394.704 rows/sec., 10.02 KiB/sec.) | ||
| 4403 | 2026.08.17 23:52:54.159317 [ 432262 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.040540289 sec. (394.669 rows/sec., 10.02 KiB/sec.) | ||
| 4404 | 2026.08.17 23:52:54.159314 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> AggregatingTransform: Aggregated. 16 to 8 rows (from 416.00 B) in 0.040535339 sec. (394.717 rows/sec., 10.02 KiB/sec.) | ||
| 4405 | 2026.08.17 23:52:54.159326 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Merging aggregated data | ||
| 4406 | 2026.08.17 23:52:54.159313 [ 434099 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47262, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 4407 | 2026.08.17 23:52:54.159337 [ 432272 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> Aggregator: Statistics updated for key=11757313445726935263: new sum_of_sizes=40, median_size=8 | ||
| 4408 | 2026.08.17 23:52:54.159354 [ 434100 ] {} <Trace> InterserverIOHTTPHandler-factory: HTTP Request for InterserverIOHTTPHandler-factory. Method: POST, Address: [::1]:47260, User-Agent: ClickHouse/23.8.7.24, Length: 0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none) | ||
| 4409 | 2026.08.17 23:52:54.159374 [ 434099 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Fmeasurements_cumulativef64_local%2Freplicas%2F1&part=all_4_4_0&client_protocol_version=8&compress=false | ||
| 4410 | 2026.08.17 23:52:54.159388 [ 434099 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Replicated PartsService): Sending part all_4_4_0 | ||
| 4411 | 2026.08.17 23:52:54.159393 [ 434100 ] {} <Trace> InterserverIOHTTPHandler: Request URI: /?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2F1%2Fmeasurements_cumulativef64_local%2Freplicas%2F1&part=all_4_4_0&client_protocol_version=8&compress=false | ||
| 4412 | 2026.08.17 23:52:54.159401 [ 434100 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (Replicated PartsService): Sending part all_4_4_0 | ||
| 4413 | 2026.08.17 23:52:54.159605 [ 434100 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 4414 | 2026.08.17 23:52:54.159612 [ 434099 ] {} <Debug> InterserverIOHTTPHandler: Done processing query | ||
| 4415 | 2026.08.17 23:52:54.159704 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> executeQuery: Read 120 rows, 8.48 KiB in 0.044363 sec., 2704.9568333972 rows/sec., 191.07 KiB/sec. | ||
| 4416 | 2026.08.17 23:52:54.159890 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Trace> HashJoin: (0x7feb314fc518) Join data is being destroyed, 12800 bytes and 32 rows in hash table | ||
| 4417 | 2026.08.17 23:52:54.160242 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> MemoryTracker: Peak memory usage (for query): 369.96 KiB. | ||
| 4418 | 2026.08.17 23:52:54.160245 [ 433548 ] {777613fa-6735-42a0-b20c-0fb99d60d9cf} <Debug> TCPHandler: Processed in 0.045008402 sec. | ||
| 4419 | 2026.08.17 23:52:54.160529 [ 433548 ] {} <Trace> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default | ||
| 4420 | 2026.08.17 23:52:54.160571 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> executeQuery: Query span trace_id for opentelemetry log: dbcdba3c-f833-72e3-11c6-292da2a4932c | ||
| 4421 | 2026.08.17 23:52:54.160660 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Debug> executeQuery: (from [::1]:56652) SELECT timeseries_key, start_time, timestamp, datum FROM oximeter.measurements_cumulativef64 WHERE timeseries_name = 'virtual_machine:cpu_busy' AND ((greater(timestamp, '2000-01-01 00:00:00.000000000') AND timeseries_key IN (423232527771204904,462795175800602962,931241130902101787,2318161133194800099,3810615656589673070,5210923722195156639,5419464027902213652,6752048178732652911,6772052453391040098,7609507504655379271,7613129139487427710,8648745482163919764,9406430691007220535,11185254949005441257,11365707485692726787,11432546062645081728,11873430000991232922,12334289709033436885,12428098974121224233,13097606626804240593,13412992881554994937,13674246925771883735,13850886821849078558,15203798703989247109,15228222593821303909,15279762139926950523,15515585004882416385,15520859351690753739,15768325978800699421,15994013491234619255,16524397771145742832,16529823099607887706))) ORDER BY timeseries_key, start_time, timestamp LIMIT 1000001 (stage: Complete) | ||
| 4422 | 2026.08.17 23:52:54.160940 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 4423 | 2026.08.17 23:52:54.161131 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64 | ||
| 4424 | 2026.08.17 23:52:54.161290 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> InterpreterSelectQuery: The min valid primary key position for moving to the tail of PREWHERE is 1 | ||
| 4425 | 2026.08.17 23:52:54.161311 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Debug> InterpreterSelectQuery: MergeTreeWhereOptimizer: condition "(timestamp > '2000-01-01 00:00:00.000000000') AND (timeseries_key IN (423232527771204904, 462795175800602962, 931241130902101787, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6752048178732652911, 6772052453391040098, 7609507504655379271, 7613129139487427710, 8648745482163919764, 9406430691007220535, 11185254949005441257, 11365707485692726787, 11432546062645081728, 11873430000991232922, 12334289709033436885, 12428098974121224233, 13097606626804240593, 13412992881554994937, 13674246925771883735, 13850886821849078558, 15203798703989247109, 15228222593821303909, 15279762139926950523, 15515585004882416385, 15520859351690753739, 15768325978800699421, 15994013491234619255, 16524397771145742832, 16529823099607887706)) AND (timeseries_name = 'virtual_machine:cpu_busy')" moved to PREWHERE | ||
| 4426 | 2026.08.17 23:52:54.161452 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> ContextAccess (default): Access granted: SELECT(timeseries_name, timeseries_key, start_time, timestamp, datum) ON oximeter.measurements_cumulativef64_local | ||
| 4427 | 2026.08.17 23:52:54.161477 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> InterpreterSelectQuery: FetchColumns -> Complete | ||
| 4428 | 2026.08.17 23:52:54.161506 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> InterpreterSelectQuery: Complete -> Complete | ||
| 4429 | 2026.08.17 23:52:54.161598 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Key condition: (column 3 in ('946684800', +Inf)), (column 1 in 32-element set), and, (column 0 in ['virtual_machine:cpu_busy', 'virtual_machine:cpu_busy']), and | ||
| 4430 | 2026.08.17 23:52:54.161648 [ 432264 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Used generic exclusion search over index for part all_4_4_0 with 1 steps | ||
| 4431 | 2026.08.17 23:52:54.161651 [ 432287 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Used generic exclusion search over index for part all_0_0_0 with 1 steps | ||
| 4432 | 2026.08.17 23:52:54.161648 [ 432265 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Used generic exclusion search over index for part all_3_3_0 with 1 steps | ||
| 4433 | 2026.08.17 23:52:54.161663 [ 432272 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Used generic exclusion search over index for part all_1_1_0 with 1 steps | ||
| 4434 | 2026.08.17 23:52:54.161709 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Debug> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Selected 4/4 parts by partition key, 4 parts by primary key, 4/4 marks by primary key, 4 marks to read from 4 ranges | ||
| 4435 | 2026.08.17 23:52:54.161714 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (SelectExecutor): Spreading ranges among streams with order | ||
| 4436 | 2026.08.17 23:52:54.161778 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 3 steps: "greater(timestamp, '2000-01-01 00:00:00.000000000')", "in(timeseries_key, (423232527771204904, 462795175800602962, 931241130902101787, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6752048178732652911, 6772052453391040098, 7609507504655379271, 7613129139487427710, 8648745482163919764, 9406430691007220535, 11185254949005441257, 11365707485692726787, 11432546062645081728, 11873430000991232922, 12334289709033436885, 12428098974121224233, 13097606626804240593, 13412992881554994937, 13674246925771883735, 13850886821849078558, 15203798703989247109, 15228222593821303909, 15279762139926950523, 15515585004882416385, 15520859351690753739, 15768325978800699421, 15994013491234619255, 16524397771145742832, 16529823099607887706))", "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4437 | 2026.08.17 23:52:54.161786 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_4_4_0, approx. 400 rows starting from 0 | ||
| 4438 | 2026.08.17 23:52:54.161820 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 3 steps: "greater(timestamp, '2000-01-01 00:00:00.000000000')", "in(timeseries_key, (423232527771204904, 462795175800602962, 931241130902101787, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6752048178732652911, 6772052453391040098, 7609507504655379271, 7613129139487427710, 8648745482163919764, 9406430691007220535, 11185254949005441257, 11365707485692726787, 11432546062645081728, 11873430000991232922, 12334289709033436885, 12428098974121224233, 13097606626804240593, 13412992881554994937, 13674246925771883735, 13850886821849078558, 15203798703989247109, 15228222593821303909, 15279762139926950523, 15515585004882416385, 15520859351690753739, 15768325978800699421, 15994013491234619255, 16524397771145742832, 16529823099607887706))", "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4439 | 2026.08.17 23:52:54.161822 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_3_3_0, approx. 400 rows starting from 0 | ||
| 4440 | 2026.08.17 23:52:54.161846 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 3 steps: "greater(timestamp, '2000-01-01 00:00:00.000000000')", "in(timeseries_key, (423232527771204904, 462795175800602962, 931241130902101787, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6752048178732652911, 6772052453391040098, 7609507504655379271, 7613129139487427710, 8648745482163919764, 9406430691007220535, 11185254949005441257, 11365707485692726787, 11432546062645081728, 11873430000991232922, 12334289709033436885, 12428098974121224233, 13097606626804240593, 13412992881554994937, 13674246925771883735, 13850886821849078558, 15203798703989247109, 15228222593821303909, 15279762139926950523, 15515585004882416385, 15520859351690753739, 15768325978800699421, 15994013491234619255, 16524397771145742832, 16529823099607887706))", "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4441 | 2026.08.17 23:52:54.161849 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_1_1_0, approx. 400 rows starting from 0 | ||
| 4442 | 2026.08.17 23:52:54.161874 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> MergeTreeBaseSelectProcessor: PREWHERE condition was split into 3 steps: "greater(timestamp, '2000-01-01 00:00:00.000000000')", "in(timeseries_key, (423232527771204904, 462795175800602962, 931241130902101787, 2318161133194800099, 3810615656589673070, 5210923722195156639, 5419464027902213652, 6752048178732652911, 6772052453391040098, 7609507504655379271, 7613129139487427710, 8648745482163919764, 9406430691007220535, 11185254949005441257, 11365707485692726787, 11432546062645081728, 11873430000991232922, 12334289709033436885, 12428098974121224233, 13097606626804240593, 13412992881554994937, 13674246925771883735, 13850886821849078558, 15203798703989247109, 15228222593821303909, 15279762139926950523, 15515585004882416385, 15520859351690753739, 15768325978800699421, 15994013491234619255, 16524397771145742832, 16529823099607887706))", "equals(timeseries_name, 'virtual_machine:cpu_busy')" | ||
| 4443 | 2026.08.17 23:52:54.161879 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part all_0_0_0, approx. 400 rows starting from 0 | ||
| 4444 | 2026.08.17 23:52:54.162444 [ 432271 ] {8477b243-5a86-4486-9348-51632d32d433} <Debug> MergingSortedTransform: Merge sorted 1 blocks, 1600 rows in 0 sec. | ||
| 4445 | 2026.08.17 23:52:54.162589 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Debug> executeQuery: Read 1600 rows, 103.13 KiB in 0.002007 sec., 797209.7658196313 rows/sec., 50.18 MiB/sec. | ||
| 4446 | 2026.08.17 23:52:54.162753 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Debug> MemoryTracker: Peak memory usage (for query): 509.64 KiB. | ||
| 4447 | 2026.08.17 23:52:54.164764 [ 433548 ] {8477b243-5a86-4486-9348-51632d32d433} <Debug> TCPHandler: Processed in 0.004281073 sec. | ||
| 4448 | 2026.08.17 23:52:54.165451 [ 432022 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000005 to queue with type MERGE_PARTS with virtual parts [all_0_4_1] | ||
| 4449 | 2026.08.17 23:52:54.165463 [ 432022 ] {} <Debug> oximeter.fields_i64_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 4450 | 2026.08.17 23:52:54.165523 [ 432225 ] {} <Trace> 2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1 (MergeFromLogEntryTask): Executing log entry to merge parts all_0_0_0, all_1_1_0, all_2_2_0, all_3_3_0, all_4_4_0 to all_0_4_1 | ||
| 4451 | 2026.08.17 23:52:54.165544 [ 432225 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4452 | 2026.08.17 23:52:54.165552 [ 432225 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.79 GiB. | ||
| 4453 | 2026.08.17 23:52:54.165606 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> MergeTask::PrepareStage: Merging 5 parts: from all_0_0_0 to all_4_4_0 into Compact with storage Full | ||
| 4454 | 2026.08.17 23:52:54.165645 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4455 | 2026.08.17 23:52:54.165654 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_0_0_0, total 8 rows starting from the beginning of the part | ||
| 4456 | 2026.08.17 23:52:54.165682 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_1_1_0, total 8 rows starting from the beginning of the part | ||
| 4457 | 2026.08.17 23:52:54.165699 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_2_2_0, total 8 rows starting from the beginning of the part | ||
| 4458 | 2026.08.17 23:52:54.165716 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_3_3_0, total 8 rows starting from the beginning of the part | ||
| 4459 | 2026.08.17 23:52:54.165743 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_4_4_0, total 8 rows starting from the beginning of the part | ||
| 4460 | 2026.08.17 23:52:54.166066 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 40 rows, containing 5 columns (5 merged, 0 gathered) in 0.000497221 sec., 80447.12512142489 rows/sec., 5.22 MiB/sec. | ||
| 4461 | 2026.08.17 23:52:54.166134 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Trace> MergedBlockOutputStream: filled checksums all_0_4_1 (state Temporary) | ||
| 4462 | 2026.08.17 23:52:54.166245 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Renaming temporary part tmp_merge_all_0_4_1 to all_0_4_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4463 | 2026.08.17 23:52:54.166264 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Merged 5 parts: [all_0_0_0, all_4_4_0] -> all_0_4_1 | ||
| 4464 | 2026.08.17 23:52:54.166276 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Committing part all_0_4_1 to zookeeper | ||
| 4465 | 2026.08.17 23:52:54.187336 [ 432225 ] {2d79911c-8882-4309-a914-7e64c31972ae::all_0_4_1} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Part all_0_4_1 committed to zookeeper | ||
| 4466 | 2026.08.17 23:52:54.193303 [ 432225 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 3.27 MiB. | ||
| 4467 | 2026.08.17 23:52:54.193348 [ 432053 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73436622848, merge_with_ttl_allowedtrue) | ||
| 4468 | 2026.08.17 23:52:54.193365 [ 432053 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 4469 | 2026.08.17 23:52:54.290061 [ 431940 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4470 | 2026.08.17 23:52:54.290067 [ 432296 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4471 | 2026.08.17 23:52:54.290063 [ 433548 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4472 | 2026.08.17 23:52:54.290087 [ 433550 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4473 | 2026.08.17 23:52:54.290094 [ 433559 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4474 | 2026.08.17 23:52:54.290088 [ 431940 ] {} <Debug> TCP-Session: a09377a6-5278-42c3-be39-a1d8738232bd Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4475 | 2026.08.17 23:52:54.290106 [ 431938 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4476 | 2026.08.17 23:52:54.290109 [ 433548 ] {} <Debug> TCP-Session: e8d323a8-6e38-43e2-8c93-e8d7976d8ddd Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4477 | 2026.08.17 23:52:54.290098 [ 432296 ] {} <Debug> TCP-Session: 0af081e4-339a-49c1-9e68-910979dcec70 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4478 | 2026.08.17 23:52:54.290116 [ 433559 ] {} <Debug> TCP-Session: 754374cb-46d0-4c7c-b0d0-ba38fa24ffe6 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4479 | 2026.08.17 23:52:54.290083 [ 433549 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4480 | 2026.08.17 23:52:54.290109 [ 433547 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4481 | 2026.08.17 23:52:54.290109 [ 433550 ] {} <Debug> TCP-Session: 4c0f0ad7-c046-4308-85fa-f07c2377a6c4 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4482 | 2026.08.17 23:52:54.290141 [ 433549 ] {} <Debug> TCP-Session: 0763c507-088e-4eec-976b-357b45555ce7 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4483 | 2026.08.17 23:52:54.290141 [ 433547 ] {} <Debug> TCP-Session: abaefab2-85c9-435d-b18d-07599f065526 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4484 | 2026.08.17 23:52:54.290126 [ 431938 ] {} <Debug> TCP-Session: 609225df-4b76-40ae-9458-3310c59c57d9 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4485 | 2026.08.17 23:52:54.309525 [ 431938 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56670 | ||
| 4486 | 2026.08.17 23:52:54.309618 [ 431938 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 4487 | 2026.08.17 23:52:54.309632 [ 431938 ] {} <Debug> TCP-Session: cdebcbab-e5c4-4cb3-a60c-75f88e97d5f4 Authenticating user 'default' from [::1]:56670 | ||
| 4488 | 2026.08.17 23:52:54.309639 [ 431938 ] {} <Debug> TCP-Session: cdebcbab-e5c4-4cb3-a60c-75f88e97d5f4 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4489 | 2026.08.17 23:52:54.309642 [ 431938 ] {} <Debug> TCP-Session: cdebcbab-e5c4-4cb3-a60c-75f88e97d5f4 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4490 | 2026.08.17 23:52:54.309942 [ 432296 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56676 | ||
| 4491 | 2026.08.17 23:52:54.309949 [ 433549 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56678 | ||
| 4492 | 2026.08.17 23:52:54.309958 [ 433550 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56682 | ||
| 4493 | 2026.08.17 23:52:54.309967 [ 431940 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56672 | ||
| 4494 | 2026.08.17 23:52:54.309963 [ 433547 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56674 | ||
| 4495 | 2026.08.17 23:52:54.309958 [ 433548 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56680 | ||
| 4496 | 2026.08.17 23:52:54.309970 [ 433559 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56684 | ||
| 4497 | 2026.08.17 23:52:54.310102 [ 434109 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:56686 | ||
| 4498 | 2026.08.17 23:52:54.310134 [ 431940 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 4499 | 2026.08.17 23:52:54.310148 [ 431940 ] {} <Debug> TCP-Session: c176683c-4716-4034-b99c-be9c0b81a117 Authenticating user 'default' from [::1]:56672 | ||
| 4500 | 2026.08.17 23:52:54.310153 [ 431940 ] {} <Debug> TCP-Session: c176683c-4716-4034-b99c-be9c0b81a117 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4501 | 2026.08.17 23:52:54.310155 [ 431940 ] {} <Debug> TCP-Session: c176683c-4716-4034-b99c-be9c0b81a117 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4502 | 2026.08.17 23:52:54.310194 [ 433547 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 4503 | 2026.08.17 23:52:54.310199 [ 433547 ] {} <Debug> TCP-Session: 572a8973-6d49-4609-a0d1-05208e38721b Authenticating user 'default' from [::1]:56674 | ||
| 4504 | 2026.08.17 23:52:54.310206 [ 433547 ] {} <Debug> TCP-Session: 572a8973-6d49-4609-a0d1-05208e38721b Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4505 | 2026.08.17 23:52:54.310208 [ 433547 ] {} <Debug> TCP-Session: 572a8973-6d49-4609-a0d1-05208e38721b Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4506 | 2026.08.17 23:52:54.310210 [ 432296 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 4507 | 2026.08.17 23:52:54.310215 [ 432296 ] {} <Debug> TCP-Session: 902aca61-3710-43ef-934f-7f03e2f089d7 Authenticating user 'default' from [::1]:56676 | ||
| 4508 | 2026.08.17 23:52:54.310218 [ 432296 ] {} <Debug> TCP-Session: 902aca61-3710-43ef-934f-7f03e2f089d7 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4509 | 2026.08.17 23:52:54.310219 [ 433549 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 4510 | 2026.08.17 23:52:54.310220 [ 432296 ] {} <Debug> TCP-Session: 902aca61-3710-43ef-934f-7f03e2f089d7 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4511 | 2026.08.17 23:52:54.310224 [ 433549 ] {} <Debug> TCP-Session: d4366754-da4c-4d54-97d4-fa58ced0962b Authenticating user 'default' from [::1]:56678 | ||
| 4512 | 2026.08.17 23:52:54.310227 [ 433549 ] {} <Debug> TCP-Session: d4366754-da4c-4d54-97d4-fa58ced0962b Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4513 | 2026.08.17 23:52:54.310228 [ 433548 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 4514 | 2026.08.17 23:52:54.310237 [ 433548 ] {} <Debug> TCP-Session: eea43647-7f2f-412f-8b36-f0b6795830df Authenticating user 'default' from [::1]:56680 | ||
| 4515 | 2026.08.17 23:52:54.310240 [ 433548 ] {} <Debug> TCP-Session: eea43647-7f2f-412f-8b36-f0b6795830df Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4516 | 2026.08.17 23:52:54.310241 [ 433550 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 4517 | 2026.08.17 23:52:54.310242 [ 433548 ] {} <Debug> TCP-Session: eea43647-7f2f-412f-8b36-f0b6795830df Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4518 | 2026.08.17 23:52:54.310247 [ 433550 ] {} <Debug> TCP-Session: a38c033b-8c7e-489f-99d2-f07d44267979 Authenticating user 'default' from [::1]:56682 | ||
| 4519 | 2026.08.17 23:52:54.310250 [ 433550 ] {} <Debug> TCP-Session: a38c033b-8c7e-489f-99d2-f07d44267979 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4520 | 2026.08.17 23:52:54.310255 [ 433550 ] {} <Debug> TCP-Session: a38c033b-8c7e-489f-99d2-f07d44267979 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4521 | 2026.08.17 23:52:54.310249 [ 433559 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 4522 | 2026.08.17 23:52:54.310265 [ 433559 ] {} <Debug> TCP-Session: d08ba94b-2b00-496a-8a74-2bc729e2120e Authenticating user 'default' from [::1]:56684 | ||
| 4523 | 2026.08.17 23:52:54.310265 [ 434109 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 4524 | 2026.08.17 23:52:54.310268 [ 433559 ] {} <Debug> TCP-Session: d08ba94b-2b00-496a-8a74-2bc729e2120e Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4525 | 2026.08.17 23:52:54.310270 [ 433559 ] {} <Debug> TCP-Session: d08ba94b-2b00-496a-8a74-2bc729e2120e Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4526 | 2026.08.17 23:52:54.310276 [ 434109 ] {} <Debug> TCP-Session: bef4accb-69fe-46d9-bd48-13f4b1b4ebb4 Authenticating user 'default' from [::1]:56686 | ||
| 4527 | 2026.08.17 23:52:54.310229 [ 433549 ] {} <Debug> TCP-Session: d4366754-da4c-4d54-97d4-fa58ced0962b Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4528 | 2026.08.17 23:52:54.310278 [ 434109 ] {} <Debug> TCP-Session: bef4accb-69fe-46d9-bd48-13f4b1b4ebb4 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4529 | 2026.08.17 23:52:54.310280 [ 434109 ] {} <Debug> TCP-Session: bef4accb-69fe-46d9-bd48-13f4b1b4ebb4 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4530 | 2026.08.17 23:52:54.310423 [ 433549 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4531 | 2026.08.17 23:52:54.310428 [ 433549 ] {} <Debug> TCP-Session: d4366754-da4c-4d54-97d4-fa58ced0962b Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4532 | 2026.08.17 23:52:55.000407 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 441.09 MiB, peak 452.19 MiB, free memory in arenas 78.82 MiB, will set to 438.06 MiB (RSS), difference: -3.04 MiB | ||
| 4533 | 2026.08.17 23:53:00.325758 [ 432124 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4534 | 2026.08.17 23:53:00.325781 [ 432124 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4535 | 2026.08.17 23:53:00.346153 [ 432254 ] {} <Trace> SystemLog (system.opentelemetry_span_log): Flushing system log, 195 entries to flush up to offset 320 | ||
| 4536 | 2026.08.17 23:53:00.347776 [ 432254 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4537 | 2026.08.17 23:53:00.347796 [ 432254 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.77 GiB. | ||
| 4538 | 2026.08.17 23:53:00.348180 [ 432254 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_3_3_0 (state Temporary) | ||
| 4539 | 2026.08.17 23:53:00.348374 [ 432254 ] {} <Trace> system.opentelemetry_span_log (0309c3c4-4bf3-4f8a-9c55-758be4e733c9): Renaming temporary part tmp_insert_202608_3_3_0 to 202608_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4540 | 2026.08.17 23:53:00.348460 [ 432254 ] {} <Trace> SystemLog (system.opentelemetry_span_log): Flushed system log up to offset 320 | ||
| 4541 | 2026.08.17 23:53:00.351799 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2578 entries to flush up to offset 11711 | ||
| 4542 | 2026.08.17 23:53:00.353067 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4543 | 2026.08.17 23:53:00.353080 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.77 GiB. | ||
| 4544 | 2026.08.17 23:53:00.353500 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_6_6_0 (state Temporary) | ||
| 4545 | 2026.08.17 23:53:00.353608 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_6_6_0 to 202608_7_7_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4546 | 2026.08.17 23:53:00.353671 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 11711 | ||
| 4547 | 2026.08.17 23:53:00.418725 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 46 | ||
| 4548 | 2026.08.17 23:53:00.426027 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4549 | 2026.08.17 23:53:00.426045 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.77 GiB. | ||
| 4550 | 2026.08.17 23:53:00.429604 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_6_6_0 (state Temporary) | ||
| 4551 | 2026.08.17 23:53:00.432325 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_6_6_0 to 202608_7_7_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4552 | 2026.08.17 23:53:00.433150 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 46 | ||
| 4553 | 2026.08.17 23:53:01.000409 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 439.18 MiB, peak 452.19 MiB, free memory in arenas 82.82 MiB, will set to 436.21 MiB (RSS), difference: -2.97 MiB | ||
| 4554 | 2026.08.17 23:53:07.853771 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2262 entries to flush up to offset 13973 | ||
| 4555 | 2026.08.17 23:53:07.854814 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4556 | 2026.08.17 23:53:07.854833 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.76 GiB. | ||
| 4557 | 2026.08.17 23:53:07.855364 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_7_7_0 (state Temporary) | ||
| 4558 | 2026.08.17 23:53:07.855513 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_7_7_0 to 202608_8_8_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4559 | 2026.08.17 23:53:07.855603 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 13973 | ||
| 4560 | 2026.08.17 23:53:07.934412 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 53 | ||
| 4561 | 2026.08.17 23:53:07.941054 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4562 | 2026.08.17 23:53:07.941076 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.76 GiB. | ||
| 4563 | 2026.08.17 23:53:07.946040 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_7_7_0 (state Temporary) | ||
| 4564 | 2026.08.17 23:53:07.948562 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_7_7_0 to 202608_8_8_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4565 | 2026.08.17 23:53:07.949377 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 53 | ||
| 4566 | 2026.08.17 23:53:15.325863 [ 432144 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4567 | 2026.08.17 23:53:15.325888 [ 432144 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4568 | 2026.08.17 23:53:15.355687 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2578 entries to flush up to offset 16551 | ||
| 4569 | 2026.08.17 23:53:15.356881 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4570 | 2026.08.17 23:53:15.356897 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.76 GiB. | ||
| 4571 | 2026.08.17 23:53:15.357398 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_8_8_0 (state Temporary) | ||
| 4572 | 2026.08.17 23:53:15.357529 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_8_8_0 to 202608_9_9_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4573 | 2026.08.17 23:53:15.357614 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 16551 | ||
| 4574 | 2026.08.17 23:53:15.449451 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 61 | ||
| 4575 | 2026.08.17 23:53:15.456516 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4576 | 2026.08.17 23:53:15.456535 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.76 GiB. | ||
| 4577 | 2026.08.17 23:53:15.460058 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_8_8_0 (state Temporary) | ||
| 4578 | 2026.08.17 23:53:15.462547 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_8_8_0 to 202608_9_9_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4579 | 2026.08.17 23:53:15.463346 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 61 | ||
| 4580 | 2026.08.17 23:53:22.857699 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2258 entries to flush up to offset 18809 | ||
| 4581 | 2026.08.17 23:53:22.858868 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4582 | 2026.08.17 23:53:22.858891 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.75 GiB. | ||
| 4583 | 2026.08.17 23:53:22.859437 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_9_9_0 (state Temporary) | ||
| 4584 | 2026.08.17 23:53:22.859606 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_9_9_0 to 202608_10_10_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4585 | 2026.08.17 23:53:22.859713 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 18809 | ||
| 4586 | 2026.08.17 23:53:22.963419 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 68 | ||
| 4587 | 2026.08.17 23:53:22.970745 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4588 | 2026.08.17 23:53:22.971104 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.75 GiB. | ||
| 4589 | 2026.08.17 23:53:22.974778 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_9_9_0 (state Temporary) | ||
| 4590 | 2026.08.17 23:53:22.977416 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_9_9_0 to 202608_10_10_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4591 | 2026.08.17 23:53:22.978225 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 68 | ||
| 4592 | 2026.08.17 23:53:30.325977 [ 432099 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4593 | 2026.08.17 23:53:30.325999 [ 432099 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4594 | 2026.08.17 23:53:30.359793 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2584 entries to flush up to offset 21393 | ||
| 4595 | 2026.08.17 23:53:30.360922 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4596 | 2026.08.17 23:53:30.360942 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.74 GiB. | ||
| 4597 | 2026.08.17 23:53:30.361339 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_10_10_0 (state Temporary) | ||
| 4598 | 2026.08.17 23:53:30.361475 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_10_10_0 to 202608_11_11_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4599 | 2026.08.17 23:53:30.361565 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 21393 | ||
| 4600 | 2026.08.17 23:53:30.361571 [ 432072 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_6_1 to 202608_11_11_0 | ||
| 4601 | 2026.08.17 23:53:30.361589 [ 432072 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4602 | 2026.08.17 23:53:30.361594 [ 432072 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.74 GiB. | ||
| 4603 | 2026.08.17 23:53:30.361748 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_6_1 to 202608_11_11_0 into Compact with storage Full | ||
| 4604 | 2026.08.17 23:53:30.363076 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4605 | 2026.08.17 23:53:30.363100 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 3 marks from part 202608_1_6_1, total 11493 rows starting from the beginning of the part | ||
| 4606 | 2026.08.17 23:53:30.363155 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_7_7_0, total 2578 rows starting from the beginning of the part | ||
| 4607 | 2026.08.17 23:53:30.363170 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_8_8_0, total 2262 rows starting from the beginning of the part | ||
| 4608 | 2026.08.17 23:53:30.363188 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_9_9_0, total 2578 rows starting from the beginning of the part | ||
| 4609 | 2026.08.17 23:53:30.363207 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_10_10_0, total 2258 rows starting from the beginning of the part | ||
| 4610 | 2026.08.17 23:53:30.363222 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_11_11_0, total 2584 rows starting from the beginning of the part | ||
| 4611 | 2026.08.17 23:53:30.366271 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 23753 rows, containing 4 columns (4 merged, 0 gathered) in 0.004585204 sec., 5180358.387543935 rows/sec., 76.66 MiB/sec. | ||
| 4612 | 2026.08.17 23:53:30.367027 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Trace> MergedBlockOutputStream: filled checksums 202608_1_11_2 (state Temporary) | ||
| 4613 | 2026.08.17 23:53:30.367191 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_11_2 to 202608_1_11_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4614 | 2026.08.17 23:53:30.367220 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_11_2} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_6_1, 202608_11_11_0] -> 202608_1_11_2 | ||
| 4615 | 2026.08.17 23:53:30.367271 [ 432224 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.39 MiB. | ||
| 4616 | 2026.08.17 23:53:30.478296 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 76 | ||
| 4617 | 2026.08.17 23:53:30.485658 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4618 | 2026.08.17 23:53:30.485676 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.74 GiB. | ||
| 4619 | 2026.08.17 23:53:30.489212 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_10_10_0 (state Temporary) | ||
| 4620 | 2026.08.17 23:53:30.491867 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_10_10_0 to 202608_11_11_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4621 | 2026.08.17 23:53:30.492020 [ 432036 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_6_1 to 202608_11_11_0 | ||
| 4622 | 2026.08.17 23:53:30.492052 [ 432036 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4623 | 2026.08.17 23:53:30.492058 [ 432036 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.74 GiB. | ||
| 4624 | 2026.08.17 23:53:30.492147 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_6_1 to 202608_11_11_0 into Compact with storage Full | ||
| 4625 | 2026.08.17 23:53:30.492823 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 76 | ||
| 4626 | 2026.08.17 23:53:30.495394 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4627 | 2026.08.17 23:53:30.495778 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_6_1, total 46 rows starting from the beginning of the part | ||
| 4628 | 2026.08.17 23:53:30.497662 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_7_7_0, total 8 rows starting from the beginning of the part | ||
| 4629 | 2026.08.17 23:53:30.499727 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_8_8_0, total 7 rows starting from the beginning of the part | ||
| 4630 | 2026.08.17 23:53:30.501664 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_9_9_0, total 8 rows starting from the beginning of the part | ||
| 4631 | 2026.08.17 23:53:30.503510 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_10_10_0, total 7 rows starting from the beginning of the part | ||
| 4632 | 2026.08.17 23:53:30.505342 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_11_11_0, total 8 rows starting from the beginning of the part | ||
| 4633 | 2026.08.17 23:53:30.531719 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 84 rows, containing 679 columns (679 merged, 0 gathered) in 0.039615968 sec., 2120.357124682653 rows/sec., 2.86 MiB/sec. | ||
| 4634 | 2026.08.17 23:53:30.532391 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Trace> MergedBlockOutputStream: filled checksums 202608_1_11_2 (state Temporary) | ||
| 4635 | 2026.08.17 23:53:30.535832 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_11_2 to 202608_1_11_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4636 | 2026.08.17 23:53:30.535870 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_11_2} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_6_1, 202608_11_11_0] -> 202608_1_11_2 | ||
| 4637 | 2026.08.17 23:53:30.536277 [ 432225 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.83 MiB. | ||
| 4638 | 2026.08.17 23:53:31.000405 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 440.39 MiB, peak 452.19 MiB, free memory in arenas 108.20 MiB, will set to 453.97 MiB (RSS), difference: 13.58 MiB | ||
| 4639 | 2026.08.17 23:53:32.024126 [ 432101 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulling 1 entries to queue: log-0000000005 - log-0000000005 | ||
| 4640 | 2026.08.17 23:53:32.030487 [ 432101 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Insert entry queue-0000000005 to queue with type MERGE_PARTS with virtual parts [all_0_4_1] | ||
| 4641 | 2026.08.17 23:53:32.030504 [ 432101 ] {} <Debug> oximeter.fields_uuid_local (ReplicatedMergeTreeQueue): Pulled 1 entries to queue. | ||
| 4642 | 2026.08.17 23:53:32.030587 [ 432215 ] {} <Trace> eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1 (MergeFromLogEntryTask): Executing log entry to merge parts all_0_0_0, all_1_1_0, all_2_2_0, all_3_3_0, all_4_4_0 to all_0_4_1 | ||
| 4643 | 2026.08.17 23:53:32.030612 [ 432215 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4644 | 2026.08.17 23:53:32.030618 [ 432215 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.73 GiB. | ||
| 4645 | 2026.08.17 23:53:32.030673 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> MergeTask::PrepareStage: Merging 5 parts: from all_0_0_0 to all_4_4_0 into Compact with storage Full | ||
| 4646 | 2026.08.17 23:53:32.030718 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4647 | 2026.08.17 23:53:32.030730 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_0_0_0, total 16 rows starting from the beginning of the part | ||
| 4648 | 2026.08.17 23:53:32.030766 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_1_1_0, total 16 rows starting from the beginning of the part | ||
| 4649 | 2026.08.17 23:53:32.030782 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_2_2_0, total 16 rows starting from the beginning of the part | ||
| 4650 | 2026.08.17 23:53:32.030797 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_3_3_0, total 16 rows starting from the beginning of the part | ||
| 4651 | 2026.08.17 23:53:32.030822 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> MergeTreeSequentialSource: Reading 2 marks from part all_4_4_0, total 16 rows starting from the beginning of the part | ||
| 4652 | 2026.08.17 23:53:32.031196 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 80 rows, containing 5 columns (5 merged, 0 gathered) in 0.000561041 sec., 142592.074376026 rows/sec., 10.95 MiB/sec. | ||
| 4653 | 2026.08.17 23:53:32.031272 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Trace> MergedBlockOutputStream: filled checksums all_0_4_1 (state Temporary) | ||
| 4654 | 2026.08.17 23:53:32.031387 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Renaming temporary part tmp_merge_all_0_4_1 to all_0_4_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4655 | 2026.08.17 23:53:32.033096 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Merged 5 parts: [all_0_0_0, all_4_4_0] -> all_0_4_1 | ||
| 4656 | 2026.08.17 23:53:32.033109 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Committing part all_0_4_1 to zookeeper | ||
| 4657 | 2026.08.17 23:53:32.043309 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Information> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): The part /clickhouse/tables/1/fields_uuid_local/replicas/2/parts/all_0_4_1 on a replica suddenly appeared, will recheck checksums | ||
| 4658 | 2026.08.17 23:53:32.043321 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Committing part all_0_4_1 to zookeeper | ||
| 4659 | 2026.08.17 23:53:32.049446 [ 432215 ] {eaa5beef-895d-41af-915b-bd6fb56613c4::all_0_4_1} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Part all_0_4_1 committed to zookeeper | ||
| 4660 | 2026.08.17 23:53:32.058446 [ 432215 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 3.28 MiB. | ||
| 4661 | 2026.08.17 23:53:32.058487 [ 432033 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73408151552, merge_with_ttl_allowedtrue) | ||
| 4662 | 2026.08.17 23:53:32.058499 [ 432033 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 4663 | 2026.08.17 23:53:37.861648 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2256 entries to flush up to offset 23649 | ||
| 4664 | 2026.08.17 23:53:37.862697 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4665 | 2026.08.17 23:53:37.862717 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.73 GiB. | ||
| 4666 | 2026.08.17 23:53:37.863121 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_11_11_0 (state Temporary) | ||
| 4667 | 2026.08.17 23:53:37.864659 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_11_11_0 to 202608_12_12_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4668 | 2026.08.17 23:53:37.864762 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 23649 | ||
| 4669 | 2026.08.17 23:53:37.992895 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 83 | ||
| 4670 | 2026.08.17 23:53:37.999623 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4671 | 2026.08.17 23:53:37.999640 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.73 GiB. | ||
| 4672 | 2026.08.17 23:53:38.003067 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_11_11_0 (state Temporary) | ||
| 4673 | 2026.08.17 23:53:38.005564 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_11_11_0 to 202608_12_12_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4674 | 2026.08.17 23:53:38.006370 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 83 | ||
| 4675 | 2026.08.17 23:53:45.326080 [ 432080 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4676 | 2026.08.17 23:53:45.326105 [ 432080 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4677 | 2026.08.17 23:53:45.364853 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2580 entries to flush up to offset 26229 | ||
| 4678 | 2026.08.17 23:53:45.366014 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4679 | 2026.08.17 23:53:45.366036 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.72 GiB. | ||
| 4680 | 2026.08.17 23:53:45.367883 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_12_12_0 (state Temporary) | ||
| 4681 | 2026.08.17 23:53:45.368025 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_12_12_0 to 202608_13_13_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4682 | 2026.08.17 23:53:45.368113 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 26229 | ||
| 4683 | 2026.08.17 23:53:45.506441 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 91 | ||
| 4684 | 2026.08.17 23:53:45.513737 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4685 | 2026.08.17 23:53:45.513754 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.72 GiB. | ||
| 4686 | 2026.08.17 23:53:45.517189 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_12_12_0 (state Temporary) | ||
| 4687 | 2026.08.17 23:53:45.519859 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_12_12_0 to 202608_13_13_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4688 | 2026.08.17 23:53:45.520829 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 91 | ||
| 4689 | 2026.08.17 23:53:46.000382 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 455.41 MiB, peak 463.44 MiB, free memory in arenas 86.89 MiB, will set to 452.91 MiB (RSS), difference: -2.50 MiB | ||
| 4690 | 2026.08.17 23:53:47.000384 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 452.91 MiB, peak 463.44 MiB, free memory in arenas 86.89 MiB, will set to 447.48 MiB (RSS), difference: -5.43 MiB | ||
| 4691 | 2026.08.17 23:53:48.000386 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 447.48 MiB, peak 463.44 MiB, free memory in arenas 86.89 MiB, will set to 445.18 MiB (RSS), difference: -2.30 MiB | ||
| 4692 | 2026.08.17 23:53:49.478465 [ 432032 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 4693 | 2026.08.17 23:53:49.478523 [ 432050 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73396639744, merge_with_ttl_allowedtrue) | ||
| 4694 | 2026.08.17 23:53:49.478542 [ 432050 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 4695 | 2026.08.17 23:53:52.868251 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 28483 | ||
| 4696 | 2026.08.17 23:53:52.869287 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4697 | 2026.08.17 23:53:52.869306 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.71 GiB. | ||
| 4698 | 2026.08.17 23:53:52.869768 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_13_13_0 (state Temporary) | ||
| 4699 | 2026.08.17 23:53:52.869912 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_13_13_0 to 202608_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4700 | 2026.08.17 23:53:52.870005 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 28483 | ||
| 4701 | 2026.08.17 23:53:53.021794 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 98 | ||
| 4702 | 2026.08.17 23:53:53.029413 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4703 | 2026.08.17 23:53:53.029436 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.71 GiB. | ||
| 4704 | 2026.08.17 23:53:53.033017 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_13_13_0 (state Temporary) | ||
| 4705 | 2026.08.17 23:53:53.035668 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_13_13_0 to 202608_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4706 | 2026.08.17 23:53:53.036495 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 98 | ||
| 4707 | 2026.08.17 23:53:54.108578 [ 432026 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73393459200, merge_with_ttl_allowedtrue) | ||
| 4708 | 2026.08.17 23:53:54.108591 [ 432026 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 4709 | 2026.08.17 23:53:54.193458 [ 432086 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73393412096, merge_with_ttl_allowedtrue) | ||
| 4710 | 2026.08.17 23:53:54.193478 [ 432086 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 4711 | 2026.08.17 23:54:00.326178 [ 432083 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4712 | 2026.08.17 23:54:00.326202 [ 432083 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4713 | 2026.08.17 23:54:00.370083 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2580 entries to flush up to offset 31063 | ||
| 4714 | 2026.08.17 23:54:00.371224 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4715 | 2026.08.17 23:54:00.371242 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.70 GiB. | ||
| 4716 | 2026.08.17 23:54:00.371616 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_14_14_0 (state Temporary) | ||
| 4717 | 2026.08.17 23:54:00.371758 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_14_14_0 to 202608_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4718 | 2026.08.17 23:54:00.371842 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 31063 | ||
| 4719 | 2026.08.17 23:54:00.536569 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 106 | ||
| 4720 | 2026.08.17 23:54:00.544054 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4721 | 2026.08.17 23:54:00.544072 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.70 GiB. | ||
| 4722 | 2026.08.17 23:54:00.547611 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_14_14_0 (state Temporary) | ||
| 4723 | 2026.08.17 23:54:00.550269 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_14_14_0 to 202608_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4724 | 2026.08.17 23:54:00.551080 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 106 | ||
| 4725 | 2026.08.17 23:54:07.871923 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2258 entries to flush up to offset 33321 | ||
| 4726 | 2026.08.17 23:54:07.873034 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4727 | 2026.08.17 23:54:07.873051 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.69 GiB. | ||
| 4728 | 2026.08.17 23:54:07.873541 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_15_15_0 (state Temporary) | ||
| 4729 | 2026.08.17 23:54:07.873676 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_15_15_0 to 202608_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4730 | 2026.08.17 23:54:07.873759 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 33321 | ||
| 4731 | 2026.08.17 23:54:07.873787 [ 432137 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_11_2 to 202608_16_16_0 | ||
| 4732 | 2026.08.17 23:54:07.873815 [ 432137 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4733 | 2026.08.17 23:54:07.873821 [ 432137 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.69 GiB. | ||
| 4734 | 2026.08.17 23:54:07.873921 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_11_2 to 202608_16_16_0 into Compact with storage Full | ||
| 4735 | 2026.08.17 23:54:07.874005 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4736 | 2026.08.17 23:54:07.874027 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 4 marks from part 202608_1_11_2, total 23753 rows starting from the beginning of the part | ||
| 4737 | 2026.08.17 23:54:07.874074 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_12_12_0, total 2256 rows starting from the beginning of the part | ||
| 4738 | 2026.08.17 23:54:07.874094 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_13_13_0, total 2580 rows starting from the beginning of the part | ||
| 4739 | 2026.08.17 23:54:07.874110 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_14_14_0, total 2254 rows starting from the beginning of the part | ||
| 4740 | 2026.08.17 23:54:07.874124 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_15_15_0, total 2580 rows starting from the beginning of the part | ||
| 4741 | 2026.08.17 23:54:07.874269 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_16_16_0, total 2258 rows starting from the beginning of the part | ||
| 4742 | 2026.08.17 23:54:07.878794 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 35681 rows, containing 4 columns (4 merged, 0 gathered) in 0.004914774 sec., 7259947.252915394 rows/sec., 106.17 MiB/sec. | ||
| 4743 | 2026.08.17 23:54:07.879061 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Trace> MergedBlockOutputStream: filled checksums 202608_1_16_3 (state Temporary) | ||
| 4744 | 2026.08.17 23:54:07.879275 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_16_3 to 202608_1_16_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4745 | 2026.08.17 23:54:07.879306 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_16_3} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_11_2, 202608_16_16_0] -> 202608_1_16_3 | ||
| 4746 | 2026.08.17 23:54:07.879362 [ 432224 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.65 MiB. | ||
| 4747 | 2026.08.17 23:54:08.000360 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 446.97 MiB, peak 463.44 MiB, free memory in arenas 80.37 MiB, will set to 445.13 MiB (RSS), difference: -1.85 MiB | ||
| 4748 | 2026.08.17 23:54:08.051153 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 113 | ||
| 4749 | 2026.08.17 23:54:08.058366 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4750 | 2026.08.17 23:54:08.058388 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.69 GiB. | ||
| 4751 | 2026.08.17 23:54:08.061936 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_15_15_0 (state Temporary) | ||
| 4752 | 2026.08.17 23:54:08.064547 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_15_15_0 to 202608_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4753 | 2026.08.17 23:54:08.064707 [ 432130 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_11_2 to 202608_16_16_0 | ||
| 4754 | 2026.08.17 23:54:08.064726 [ 432130 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4755 | 2026.08.17 23:54:08.064732 [ 432130 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.69 GiB. | ||
| 4756 | 2026.08.17 23:54:08.064875 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_11_2 to 202608_16_16_0 into Compact with storage Full | ||
| 4757 | 2026.08.17 23:54:08.065466 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 113 | ||
| 4758 | 2026.08.17 23:54:08.067709 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4759 | 2026.08.17 23:54:08.068095 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_11_2, total 84 rows starting from the beginning of the part | ||
| 4760 | 2026.08.17 23:54:08.069984 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_12_12_0, total 7 rows starting from the beginning of the part | ||
| 4761 | 2026.08.17 23:54:08.071830 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_13_13_0, total 8 rows starting from the beginning of the part | ||
| 4762 | 2026.08.17 23:54:08.073744 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_14_14_0, total 7 rows starting from the beginning of the part | ||
| 4763 | 2026.08.17 23:54:08.075608 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_15_15_0, total 8 rows starting from the beginning of the part | ||
| 4764 | 2026.08.17 23:54:08.077416 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_16_16_0, total 7 rows starting from the beginning of the part | ||
| 4765 | 2026.08.17 23:54:08.105271 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 121 rows, containing 679 columns (679 merged, 0 gathered) in 0.040461379 sec., 2990.50608235572 rows/sec., 3.36 MiB/sec. | ||
| 4766 | 2026.08.17 23:54:08.106561 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Trace> MergedBlockOutputStream: filled checksums 202608_1_16_3 (state Temporary) | ||
| 4767 | 2026.08.17 23:54:08.110545 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_16_3 to 202608_1_16_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4768 | 2026.08.17 23:54:08.110600 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_16_3} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_11_2, 202608_16_16_0] -> 202608_1_16_3 | ||
| 4769 | 2026.08.17 23:54:08.111257 [ 432215 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.92 MiB. | ||
| 4770 | 2026.08.17 23:54:09.000366 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 446.59 MiB, peak 463.44 MiB, free memory in arenas 100.19 MiB, will set to 464.18 MiB (RSS), difference: 17.59 MiB | ||
| 4771 | 2026.08.17 23:54:15.326281 [ 432044 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4772 | 2026.08.17 23:54:15.326303 [ 432044 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4773 | 2026.08.17 23:54:15.373839 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2580 entries to flush up to offset 35901 | ||
| 4774 | 2026.08.17 23:54:15.375068 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4775 | 2026.08.17 23:54:15.375087 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.68 GiB. | ||
| 4776 | 2026.08.17 23:54:15.375612 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_16_16_0 (state Temporary) | ||
| 4777 | 2026.08.17 23:54:15.375744 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_16_16_0 to 202608_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4778 | 2026.08.17 23:54:15.375827 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 35901 | ||
| 4779 | 2026.08.17 23:54:15.565538 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 121 | ||
| 4780 | 2026.08.17 23:54:15.572480 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4781 | 2026.08.17 23:54:15.572504 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.68 GiB. | ||
| 4782 | 2026.08.17 23:54:15.575935 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_16_16_0 (state Temporary) | ||
| 4783 | 2026.08.17 23:54:15.578466 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_16_16_0 to 202608_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4784 | 2026.08.17 23:54:15.579283 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 121 | ||
| 4785 | 2026.08.17 23:54:16.000759 [ 432270 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 8.098e-05 sec. | ||
| 4786 | 2026.08.17 23:54:22.875919 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2264 entries to flush up to offset 38165 | ||
| 4787 | 2026.08.17 23:54:22.876938 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4788 | 2026.08.17 23:54:22.876956 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.67 GiB. | ||
| 4789 | 2026.08.17 23:54:22.877364 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_17_17_0 (state Temporary) | ||
| 4790 | 2026.08.17 23:54:22.877500 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_17_17_0 to 202608_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4791 | 2026.08.17 23:54:22.877586 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 38165 | ||
| 4792 | 2026.08.17 23:54:23.079357 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 128 | ||
| 4793 | 2026.08.17 23:54:23.086673 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4794 | 2026.08.17 23:54:23.086692 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.67 GiB. | ||
| 4795 | 2026.08.17 23:54:23.090202 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_17_17_0 (state Temporary) | ||
| 4796 | 2026.08.17 23:54:23.093456 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_17_17_0 to 202608_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4797 | 2026.08.17 23:54:23.094255 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 128 | ||
| 4798 | 2026.08.17 23:54:24.322732 [ 431938 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4799 | 2026.08.17 23:54:24.322742 [ 431940 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4800 | 2026.08.17 23:54:24.322753 [ 431938 ] {} <Debug> TCP-Session: cdebcbab-e5c4-4cb3-a60c-75f88e97d5f4 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4801 | 2026.08.17 23:54:24.322753 [ 431940 ] {} <Debug> TCP-Session: c176683c-4716-4034-b99c-be9c0b81a117 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4802 | 2026.08.17 23:54:24.322760 [ 433548 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4803 | 2026.08.17 23:54:24.322753 [ 432296 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4804 | 2026.08.17 23:54:24.322774 [ 434109 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4805 | 2026.08.17 23:54:24.322767 [ 433559 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4806 | 2026.08.17 23:54:24.322783 [ 433548 ] {} <Debug> TCP-Session: eea43647-7f2f-412f-8b36-f0b6795830df Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4807 | 2026.08.17 23:54:24.322788 [ 434109 ] {} <Debug> TCP-Session: bef4accb-69fe-46d9-bd48-13f4b1b4ebb4 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4808 | 2026.08.17 23:54:24.322791 [ 432296 ] {} <Debug> TCP-Session: 902aca61-3710-43ef-934f-7f03e2f089d7 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4809 | 2026.08.17 23:54:24.322795 [ 433559 ] {} <Debug> TCP-Session: d08ba94b-2b00-496a-8a74-2bc729e2120e Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4810 | 2026.08.17 23:54:24.322825 [ 433547 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4811 | 2026.08.17 23:54:24.322834 [ 433547 ] {} <Debug> TCP-Session: 572a8973-6d49-4609-a0d1-05208e38721b Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4812 | 2026.08.17 23:54:24.322836 [ 433550 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 4813 | 2026.08.17 23:54:24.322846 [ 433550 ] {} <Debug> TCP-Session: a38c033b-8c7e-489f-99d2-f07d44267979 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 4814 | 2026.08.17 23:54:30.326412 [ 432076 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4815 | 2026.08.17 23:54:30.326441 [ 432076 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4816 | 2026.08.17 23:54:30.377692 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2578 entries to flush up to offset 40743 | ||
| 4817 | 2026.08.17 23:54:30.379781 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4818 | 2026.08.17 23:54:30.379807 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4819 | 2026.08.17 23:54:30.380543 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_18_18_0 (state Temporary) | ||
| 4820 | 2026.08.17 23:54:30.380752 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_18_18_0 to 202608_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4821 | 2026.08.17 23:54:30.380878 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 40743 | ||
| 4822 | 2026.08.17 23:54:30.594342 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 136 | ||
| 4823 | 2026.08.17 23:54:30.601253 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4824 | 2026.08.17 23:54:30.601270 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4825 | 2026.08.17 23:54:30.604654 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_18_18_0 (state Temporary) | ||
| 4826 | 2026.08.17 23:54:30.607107 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_18_18_0 to 202608_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4827 | 2026.08.17 23:54:30.607902 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 136 | ||
| 4828 | 2026.08.17 23:54:31.000257 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 466.90 MiB, peak 474.93 MiB, free memory in arenas 118.17 MiB, will set to 465.89 MiB (RSS), difference: -1.01 MiB | ||
| 4829 | 2026.08.17 23:54:32.058620 [ 432084 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73367949312, merge_with_ttl_allowedtrue) | ||
| 4830 | 2026.08.17 23:54:32.058635 [ 432084 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 4831 | 2026.08.17 23:54:37.880987 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 42997 | ||
| 4832 | 2026.08.17 23:54:37.882991 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4833 | 2026.08.17 23:54:37.883015 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4834 | 2026.08.17 23:54:37.883812 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_19_19_0 (state Temporary) | ||
| 4835 | 2026.08.17 23:54:37.884034 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_19_19_0 to 202608_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4836 | 2026.08.17 23:54:37.884157 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 42997 | ||
| 4837 | 2026.08.17 23:54:38.107988 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 143 | ||
| 4838 | 2026.08.17 23:54:38.115301 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4839 | 2026.08.17 23:54:38.115317 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4840 | 2026.08.17 23:54:38.118676 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_19_19_0 (state Temporary) | ||
| 4841 | 2026.08.17 23:54:38.121144 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_19_19_0 to 202608_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4842 | 2026.08.17 23:54:38.121910 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 143 | ||
| 4843 | 2026.08.17 23:54:45.326534 [ 432038 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4844 | 2026.08.17 23:54:45.326560 [ 432038 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4845 | 2026.08.17 23:54:45.384252 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 45573 | ||
| 4846 | 2026.08.17 23:54:45.385820 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4847 | 2026.08.17 23:54:45.385837 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4848 | 2026.08.17 23:54:45.386462 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_20_20_0 (state Temporary) | ||
| 4849 | 2026.08.17 23:54:45.386642 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_20_20_0 to 202608_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4850 | 2026.08.17 23:54:45.386738 [ 432079 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_16_3 to 202608_21_21_0 | ||
| 4851 | 2026.08.17 23:54:45.386745 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 45573 | ||
| 4852 | 2026.08.17 23:54:45.386770 [ 432079 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4853 | 2026.08.17 23:54:45.386778 [ 432079 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4854 | 2026.08.17 23:54:45.386861 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_16_3 to 202608_21_21_0 into Compact with storage Full | ||
| 4855 | 2026.08.17 23:54:45.386918 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4856 | 2026.08.17 23:54:45.386933 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 6 marks from part 202608_1_16_3, total 35681 rows starting from the beginning of the part | ||
| 4857 | 2026.08.17 23:54:45.386979 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_17_17_0, total 2580 rows starting from the beginning of the part | ||
| 4858 | 2026.08.17 23:54:45.386998 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_18_18_0, total 2264 rows starting from the beginning of the part | ||
| 4859 | 2026.08.17 23:54:45.387018 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_19_19_0, total 2578 rows starting from the beginning of the part | ||
| 4860 | 2026.08.17 23:54:45.387040 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_20_20_0, total 2254 rows starting from the beginning of the part | ||
| 4861 | 2026.08.17 23:54:45.387059 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_21_21_0, total 2576 rows starting from the beginning of the part | ||
| 4862 | 2026.08.17 23:54:45.392663 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 47933 rows, containing 4 columns (4 merged, 0 gathered) in 0.005839514 sec., 8208388.57480263 rows/sec., 119.42 MiB/sec. | ||
| 4863 | 2026.08.17 23:54:45.392884 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Trace> MergedBlockOutputStream: filled checksums 202608_1_21_4 (state Temporary) | ||
| 4864 | 2026.08.17 23:54:45.393075 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_21_4 to 202608_1_21_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4865 | 2026.08.17 23:54:45.393102 [ 432197 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_21_4} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_16_3, 202608_21_21_0] -> 202608_1_21_4 | ||
| 4866 | 2026.08.17 23:54:45.393151 [ 432197 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 4867 | 2026.08.17 23:54:45.621994 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 151 | ||
| 4868 | 2026.08.17 23:54:45.629867 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4869 | 2026.08.17 23:54:45.629882 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4870 | 2026.08.17 23:54:45.633360 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_20_20_0 (state Temporary) | ||
| 4871 | 2026.08.17 23:54:45.635891 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_20_20_0 to 202608_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4872 | 2026.08.17 23:54:45.636029 [ 432103 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_16_3 to 202608_21_21_0 | ||
| 4873 | 2026.08.17 23:54:45.636043 [ 432103 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4874 | 2026.08.17 23:54:45.636048 [ 432103 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4875 | 2026.08.17 23:54:45.636189 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_16_3 to 202608_21_21_0 into Compact with storage Full | ||
| 4876 | 2026.08.17 23:54:45.636678 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 151 | ||
| 4877 | 2026.08.17 23:54:45.639760 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4878 | 2026.08.17 23:54:45.640139 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_16_3, total 121 rows starting from the beginning of the part | ||
| 4879 | 2026.08.17 23:54:45.641981 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_17_17_0, total 8 rows starting from the beginning of the part | ||
| 4880 | 2026.08.17 23:54:45.643874 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_18_18_0, total 7 rows starting from the beginning of the part | ||
| 4881 | 2026.08.17 23:54:45.645748 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_19_19_0, total 8 rows starting from the beginning of the part | ||
| 4882 | 2026.08.17 23:54:45.647553 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_20_20_0, total 7 rows starting from the beginning of the part | ||
| 4883 | 2026.08.17 23:54:45.649341 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_21_21_0, total 8 rows starting from the beginning of the part | ||
| 4884 | 2026.08.17 23:54:45.651562 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Trace> SortDescription: Compile sort description Date 1 1DateTime 1 1 | ||
| 4885 | 2026.08.17 23:54:45.680438 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 159 rows, containing 679 columns (679 merged, 0 gathered) in 0.044312161 sec., 3588.179777555872 rows/sec., 3.64 MiB/sec. | ||
| 4886 | 2026.08.17 23:54:45.681525 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Trace> MergedBlockOutputStream: filled checksums 202608_1_21_4 (state Temporary) | ||
| 4887 | 2026.08.17 23:54:45.685075 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_21_4 to 202608_1_21_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4888 | 2026.08.17 23:54:45.685113 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_21_4} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_16_3, 202608_21_21_0] -> 202608_1_21_4 | ||
| 4889 | 2026.08.17 23:54:45.685530 [ 432216 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.99 MiB. | ||
| 4890 | 2026.08.17 23:54:46.000283 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 467.91 MiB, peak 478.65 MiB, free memory in arenas 123.48 MiB, will set to 471.41 MiB (RSS), difference: 3.51 MiB | ||
| 4891 | 2026.08.17 23:54:49.478548 [ 432053 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 4892 | 2026.08.17 23:54:49.478595 [ 432015 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73366964224, merge_with_ttl_allowedtrue) | ||
| 4893 | 2026.08.17 23:54:49.478618 [ 432015 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 4894 | 2026.08.17 23:54:52.886850 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 47827 | ||
| 4895 | 2026.08.17 23:54:52.888754 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4896 | 2026.08.17 23:54:52.888775 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4897 | 2026.08.17 23:54:52.889452 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_21_21_0 (state Temporary) | ||
| 4898 | 2026.08.17 23:54:52.889635 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_21_21_0 to 202608_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4899 | 2026.08.17 23:54:52.889742 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 47827 | ||
| 4900 | 2026.08.17 23:54:53.000191 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 471.41 MiB, peak 479.62 MiB, free memory in arenas 122.73 MiB, will set to 470.21 MiB (RSS), difference: -1.21 MiB | ||
| 4901 | 2026.08.17 23:54:53.136767 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 158 | ||
| 4902 | 2026.08.17 23:54:53.144945 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4903 | 2026.08.17 23:54:53.144962 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4904 | 2026.08.17 23:54:53.148407 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_21_21_0 (state Temporary) | ||
| 4905 | 2026.08.17 23:54:53.151000 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_21_21_0 to 202608_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4906 | 2026.08.17 23:54:53.151808 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 158 | ||
| 4907 | 2026.08.17 23:54:54.000257 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 470.91 MiB, peak 479.62 MiB, free memory in arenas 113.75 MiB, will set to 468.95 MiB (RSS), difference: -1.96 MiB | ||
| 4908 | 2026.08.17 23:54:54.108719 [ 432086 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73366493184, merge_with_ttl_allowedtrue) | ||
| 4909 | 2026.08.17 23:54:54.108736 [ 432086 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 4910 | 2026.08.17 23:54:54.193592 [ 432017 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73366493184, merge_with_ttl_allowedtrue) | ||
| 4911 | 2026.08.17 23:54:54.193610 [ 432017 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 4912 | 2026.08.17 23:55:00.326668 [ 432076 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4913 | 2026.08.17 23:55:00.326693 [ 432076 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4914 | 2026.08.17 23:55:00.389987 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2578 entries to flush up to offset 50405 | ||
| 4915 | 2026.08.17 23:55:00.391857 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4916 | 2026.08.17 23:55:00.391881 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4917 | 2026.08.17 23:55:00.392630 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_22_22_0 (state Temporary) | ||
| 4918 | 2026.08.17 23:55:00.392834 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_22_22_0 to 202608_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4919 | 2026.08.17 23:55:00.392961 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 50405 | ||
| 4920 | 2026.08.17 23:55:00.651894 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 166 | ||
| 4921 | 2026.08.17 23:55:00.658896 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4922 | 2026.08.17 23:55:00.658911 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.66 GiB. | ||
| 4923 | 2026.08.17 23:55:00.662279 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_22_22_0 (state Temporary) | ||
| 4924 | 2026.08.17 23:55:00.664871 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_22_22_0 to 202608_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4925 | 2026.08.17 23:55:00.665655 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 166 | ||
| 4926 | 2026.08.17 23:55:07.893067 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 52659 | ||
| 4927 | 2026.08.17 23:55:07.894754 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4928 | 2026.08.17 23:55:07.894776 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 4929 | 2026.08.17 23:55:07.895482 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_23_23_0 (state Temporary) | ||
| 4930 | 2026.08.17 23:55:07.895699 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_23_23_0 to 202608_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4931 | 2026.08.17 23:55:07.895822 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 52659 | ||
| 4932 | 2026.08.17 23:55:08.165743 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 173 | ||
| 4933 | 2026.08.17 23:55:08.173245 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4934 | 2026.08.17 23:55:08.173260 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 4935 | 2026.08.17 23:55:08.176726 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_23_23_0 (state Temporary) | ||
| 4936 | 2026.08.17 23:55:08.179322 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_23_23_0 to 202608_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4937 | 2026.08.17 23:55:08.180089 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 173 | ||
| 4938 | 2026.08.17 23:55:15.326785 [ 432112 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 4939 | 2026.08.17 23:55:15.326815 [ 432112 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 4940 | 2026.08.17 23:55:15.396093 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 55235 | ||
| 4941 | 2026.08.17 23:55:15.397964 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4942 | 2026.08.17 23:55:15.397984 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 4943 | 2026.08.17 23:55:15.398741 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_24_24_0 (state Temporary) | ||
| 4944 | 2026.08.17 23:55:15.398944 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_24_24_0 to 202608_25_25_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4945 | 2026.08.17 23:55:15.399065 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 55235 | ||
| 4946 | 2026.08.17 23:55:15.680174 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 181 | ||
| 4947 | 2026.08.17 23:55:15.688149 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4948 | 2026.08.17 23:55:15.688165 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 4949 | 2026.08.17 23:55:15.691622 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_24_24_0 (state Temporary) | ||
| 4950 | 2026.08.17 23:55:15.694234 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_24_24_0 to 202608_25_25_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4951 | 2026.08.17 23:55:15.694996 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 181 | ||
| 4952 | 2026.08.17 23:55:16.000240 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 473.14 MiB, peak 481.17 MiB, free memory in arenas 113.68 MiB, will set to 471.18 MiB (RSS), difference: -1.96 MiB | ||
| 4953 | 2026.08.17 23:55:22.899175 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 57489 | ||
| 4954 | 2026.08.17 23:55:22.900920 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4955 | 2026.08.17 23:55:22.900941 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 4956 | 2026.08.17 23:55:22.901624 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_25_25_0 (state Temporary) | ||
| 4957 | 2026.08.17 23:55:22.901827 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_25_25_0 to 202608_26_26_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4958 | 2026.08.17 23:55:22.901952 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 57489 | ||
| 4959 | 2026.08.17 23:55:22.901955 [ 432097 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_21_4 to 202608_26_26_0 | ||
| 4960 | 2026.08.17 23:55:22.901981 [ 432097 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4961 | 2026.08.17 23:55:22.901990 [ 432097 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 4962 | 2026.08.17 23:55:22.902151 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_21_4 to 202608_26_26_0 into Compact with storage Full | ||
| 4963 | 2026.08.17 23:55:22.902230 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4964 | 2026.08.17 23:55:22.902252 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 7 marks from part 202608_1_21_4, total 47933 rows starting from the beginning of the part | ||
| 4965 | 2026.08.17 23:55:22.902295 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_22_22_0, total 2254 rows starting from the beginning of the part | ||
| 4966 | 2026.08.17 23:55:22.902311 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_23_23_0, total 2578 rows starting from the beginning of the part | ||
| 4967 | 2026.08.17 23:55:22.902327 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_24_24_0, total 2254 rows starting from the beginning of the part | ||
| 4968 | 2026.08.17 23:55:22.902345 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_25_25_0, total 2576 rows starting from the beginning of the part | ||
| 4969 | 2026.08.17 23:55:22.902361 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_26_26_0, total 2254 rows starting from the beginning of the part | ||
| 4970 | 2026.08.17 23:55:22.907898 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 59849 rows, containing 4 columns (4 merged, 0 gathered) in 0.005815864 sec., 10290646.411264088 rows/sec., 149.16 MiB/sec. | ||
| 4971 | 2026.08.17 23:55:22.908052 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Trace> MergedBlockOutputStream: filled checksums 202608_1_26_5 (state Temporary) | ||
| 4972 | 2026.08.17 23:55:22.908188 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_26_5 to 202608_1_26_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4973 | 2026.08.17 23:55:22.908214 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_26_5} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_21_4, 202608_26_26_0] -> 202608_1_26_5 | ||
| 4974 | 2026.08.17 23:55:22.908259 [ 432198 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.67 MiB. | ||
| 4975 | 2026.08.17 23:55:23.000173 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 471.23 MiB, peak 481.17 MiB, free memory in arenas 118.89 MiB, will set to 474.55 MiB (RSS), difference: 3.33 MiB | ||
| 4976 | 2026.08.17 23:55:23.195082 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 188 | ||
| 4977 | 2026.08.17 23:55:23.203465 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4978 | 2026.08.17 23:55:23.203481 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 4979 | 2026.08.17 23:55:23.206900 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_25_25_0 (state Temporary) | ||
| 4980 | 2026.08.17 23:55:23.209538 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_25_25_0 to 202608_26_26_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4981 | 2026.08.17 23:55:23.209700 [ 432022 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_21_4 to 202608_26_26_0 | ||
| 4982 | 2026.08.17 23:55:23.209720 [ 432022 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 4983 | 2026.08.17 23:55:23.209727 [ 432022 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 4984 | 2026.08.17 23:55:23.209868 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_21_4 to 202608_26_26_0 into Compact with storage Full | ||
| 4985 | 2026.08.17 23:55:23.210302 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 188 | ||
| 4986 | 2026.08.17 23:55:23.212633 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 4987 | 2026.08.17 23:55:23.213004 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_21_4, total 159 rows starting from the beginning of the part | ||
| 4988 | 2026.08.17 23:55:23.214870 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_22_22_0, total 7 rows starting from the beginning of the part | ||
| 4989 | 2026.08.17 23:55:23.216730 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_23_23_0, total 8 rows starting from the beginning of the part | ||
| 4990 | 2026.08.17 23:55:23.218636 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_24_24_0, total 7 rows starting from the beginning of the part | ||
| 4991 | 2026.08.17 23:55:23.220457 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_25_25_0, total 8 rows starting from the beginning of the part | ||
| 4992 | 2026.08.17 23:55:23.222293 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_26_26_0, total 7 rows starting from the beginning of the part | ||
| 4993 | 2026.08.17 23:55:23.249086 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 196 rows, containing 679 columns (679 merged, 0 gathered) in 0.039273328 sec., 4990.664402059331 rows/sec., 4.43 MiB/sec. | ||
| 4994 | 2026.08.17 23:55:23.249803 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Trace> MergedBlockOutputStream: filled checksums 202608_1_26_5 (state Temporary) | ||
| 4995 | 2026.08.17 23:55:23.253233 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_26_5 to 202608_1_26_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 4996 | 2026.08.17 23:55:23.253267 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_26_5} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_21_4, 202608_26_26_0] -> 202608_1_26_5 | ||
| 4997 | 2026.08.17 23:55:23.253650 [ 432210 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.52 MiB. | ||
| 4998 | 2026.08.17 23:55:24.000256 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 476.04 MiB, peak 486.79 MiB, free memory in arenas 135.05 MiB, will set to 488.39 MiB (RSS), difference: 12.35 MiB | ||
| 4999 | 2026.08.17 23:55:30.326907 [ 432039 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5000 | 2026.08.17 23:55:30.326937 [ 432039 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5001 | 2026.08.17 23:55:30.402055 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 60065 | ||
| 5002 | 2026.08.17 23:55:30.404000 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5003 | 2026.08.17 23:55:30.404023 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5004 | 2026.08.17 23:55:30.404594 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_26_26_0 (state Temporary) | ||
| 5005 | 2026.08.17 23:55:30.404801 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_26_26_0 to 202608_27_27_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5006 | 2026.08.17 23:55:30.404926 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 60065 | ||
| 5007 | 2026.08.17 23:55:30.710387 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 196 | ||
| 5008 | 2026.08.17 23:55:30.718554 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5009 | 2026.08.17 23:55:30.718569 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5010 | 2026.08.17 23:55:30.722030 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_26_26_0 (state Temporary) | ||
| 5011 | 2026.08.17 23:55:30.724685 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_26_26_0 to 202608_27_27_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5012 | 2026.08.17 23:55:30.725458 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 196 | ||
| 5013 | 2026.08.17 23:55:32.058760 [ 432090 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73364316160, merge_with_ttl_allowedtrue) | ||
| 5014 | 2026.08.17 23:55:32.058785 [ 432090 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 5015 | 2026.08.17 23:55:37.905035 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 62319 | ||
| 5016 | 2026.08.17 23:55:37.906893 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5017 | 2026.08.17 23:55:37.906916 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5018 | 2026.08.17 23:55:37.907682 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_27_27_0 (state Temporary) | ||
| 5019 | 2026.08.17 23:55:37.907889 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_27_27_0 to 202608_28_28_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5020 | 2026.08.17 23:55:37.908017 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 62319 | ||
| 5021 | 2026.08.17 23:55:38.225545 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 203 | ||
| 5022 | 2026.08.17 23:55:38.232955 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5023 | 2026.08.17 23:55:38.232972 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5024 | 2026.08.17 23:55:38.236429 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_27_27_0 (state Temporary) | ||
| 5025 | 2026.08.17 23:55:38.239051 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_27_27_0 to 202608_28_28_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5026 | 2026.08.17 23:55:38.239837 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 203 | ||
| 5027 | 2026.08.17 23:55:45.327030 [ 432158 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5028 | 2026.08.17 23:55:45.327054 [ 432158 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5029 | 2026.08.17 23:55:45.408118 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 64895 | ||
| 5030 | 2026.08.17 23:55:45.410065 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5031 | 2026.08.17 23:55:45.410086 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5032 | 2026.08.17 23:55:45.410807 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_28_28_0 (state Temporary) | ||
| 5033 | 2026.08.17 23:55:45.411010 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_28_28_0 to 202608_29_29_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5034 | 2026.08.17 23:55:45.411129 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 64895 | ||
| 5035 | 2026.08.17 23:55:45.739915 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 211 | ||
| 5036 | 2026.08.17 23:55:45.747253 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5037 | 2026.08.17 23:55:45.747268 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5038 | 2026.08.17 23:55:45.750751 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_28_28_0 (state Temporary) | ||
| 5039 | 2026.08.17 23:55:45.753392 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_28_28_0 to 202608_29_29_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5040 | 2026.08.17 23:55:45.754160 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 211 | ||
| 5041 | 2026.08.17 23:55:49.478630 [ 432059 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 5042 | 2026.08.17 23:55:49.478656 [ 432059 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73363603456, merge_with_ttl_allowedtrue) | ||
| 5043 | 2026.08.17 23:55:49.478663 [ 432059 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 5044 | 2026.08.17 23:55:52.911263 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 67149 | ||
| 5045 | 2026.08.17 23:55:52.913121 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5046 | 2026.08.17 23:55:52.913144 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5047 | 2026.08.17 23:55:52.913900 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_29_29_0 (state Temporary) | ||
| 5048 | 2026.08.17 23:55:52.914114 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_29_29_0 to 202608_30_30_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5049 | 2026.08.17 23:55:52.914252 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 67149 | ||
| 5050 | 2026.08.17 23:55:53.254241 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 218 | ||
| 5051 | 2026.08.17 23:55:53.262370 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5052 | 2026.08.17 23:55:53.262387 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5053 | 2026.08.17 23:55:53.265853 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_29_29_0 (state Temporary) | ||
| 5054 | 2026.08.17 23:55:53.268504 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_29_29_0 to 202608_30_30_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5055 | 2026.08.17 23:55:53.269260 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 218 | ||
| 5056 | 2026.08.17 23:55:54.108864 [ 432096 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73363271680, merge_with_ttl_allowedtrue) | ||
| 5057 | 2026.08.17 23:55:54.108884 [ 432096 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 5058 | 2026.08.17 23:55:54.193739 [ 432024 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73363271680, merge_with_ttl_allowedtrue) | ||
| 5059 | 2026.08.17 23:55:54.193775 [ 432024 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 5060 | 2026.08.17 23:56:00.327164 [ 432086 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5061 | 2026.08.17 23:56:00.327198 [ 432086 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5062 | 2026.08.17 23:56:00.414352 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 69725 | ||
| 5063 | 2026.08.17 23:56:00.416192 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5064 | 2026.08.17 23:56:00.416214 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5065 | 2026.08.17 23:56:00.416890 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_30_30_0 (state Temporary) | ||
| 5066 | 2026.08.17 23:56:00.417080 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_30_30_0 to 202608_31_31_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5067 | 2026.08.17 23:56:00.417191 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 69725 | ||
| 5068 | 2026.08.17 23:56:00.417190 [ 432017 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_26_5 to 202608_31_31_0 | ||
| 5069 | 2026.08.17 23:56:00.417222 [ 432017 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5070 | 2026.08.17 23:56:00.417228 [ 432017 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5071 | 2026.08.17 23:56:00.417317 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_26_5 to 202608_31_31_0 into Compact with storage Full | ||
| 5072 | 2026.08.17 23:56:00.417390 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5073 | 2026.08.17 23:56:00.417409 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 9 marks from part 202608_1_26_5, total 59849 rows starting from the beginning of the part | ||
| 5074 | 2026.08.17 23:56:00.417447 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_27_27_0, total 2576 rows starting from the beginning of the part | ||
| 5075 | 2026.08.17 23:56:00.417466 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_28_28_0, total 2254 rows starting from the beginning of the part | ||
| 5076 | 2026.08.17 23:56:00.417484 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_29_29_0, total 2576 rows starting from the beginning of the part | ||
| 5077 | 2026.08.17 23:56:00.417500 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_30_30_0, total 2254 rows starting from the beginning of the part | ||
| 5078 | 2026.08.17 23:56:00.417520 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_31_31_0, total 2576 rows starting from the beginning of the part | ||
| 5079 | 2026.08.17 23:56:00.423288 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 72085 rows, containing 4 columns (4 merged, 0 gathered) in 0.006010724 sec., 11992731.657617284 rows/sec., 173.50 MiB/sec. | ||
| 5080 | 2026.08.17 23:56:00.423460 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Trace> MergedBlockOutputStream: filled checksums 202608_1_31_6 (state Temporary) | ||
| 5081 | 2026.08.17 23:56:00.423594 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_31_6 to 202608_1_31_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5082 | 2026.08.17 23:56:00.423618 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_31_6} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_26_5, 202608_31_31_0] -> 202608_1_31_6 | ||
| 5083 | 2026.08.17 23:56:00.423660 [ 432224 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 5084 | 2026.08.17 23:56:00.769344 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 226 | ||
| 5085 | 2026.08.17 23:56:00.777381 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5086 | 2026.08.17 23:56:00.777396 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5087 | 2026.08.17 23:56:00.780795 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_30_30_0 (state Temporary) | ||
| 5088 | 2026.08.17 23:56:00.783412 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_30_30_0 to 202608_31_31_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5089 | 2026.08.17 23:56:00.783542 [ 432058 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_26_5 to 202608_31_31_0 | ||
| 5090 | 2026.08.17 23:56:00.783554 [ 432058 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5091 | 2026.08.17 23:56:00.783558 [ 432058 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5092 | 2026.08.17 23:56:00.783675 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_26_5 to 202608_31_31_0 into Compact with storage Full | ||
| 5093 | 2026.08.17 23:56:00.784207 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 226 | ||
| 5094 | 2026.08.17 23:56:00.788919 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5095 | 2026.08.17 23:56:00.789536 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_26_5, total 196 rows starting from the beginning of the part | ||
| 5096 | 2026.08.17 23:56:00.792416 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_27_27_0, total 8 rows starting from the beginning of the part | ||
| 5097 | 2026.08.17 23:56:00.795017 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_28_28_0, total 7 rows starting from the beginning of the part | ||
| 5098 | 2026.08.17 23:56:00.797506 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_29_29_0, total 8 rows starting from the beginning of the part | ||
| 5099 | 2026.08.17 23:56:00.799753 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_30_30_0, total 7 rows starting from the beginning of the part | ||
| 5100 | 2026.08.17 23:56:00.801882 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_31_31_0, total 8 rows starting from the beginning of the part | ||
| 5101 | 2026.08.17 23:56:00.828897 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 234 rows, containing 679 columns (679 merged, 0 gathered) in 0.045272982 sec., 5168.645617379478 rows/sec., 4.13 MiB/sec. | ||
| 5102 | 2026.08.17 23:56:00.829627 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Trace> MergedBlockOutputStream: filled checksums 202608_1_31_6 (state Temporary) | ||
| 5103 | 2026.08.17 23:56:00.832961 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_31_6 to 202608_1_31_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5104 | 2026.08.17 23:56:00.832995 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_31_6} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_26_5, 202608_31_31_0] -> 202608_1_31_6 | ||
| 5105 | 2026.08.17 23:56:00.833392 [ 432225 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.50 MiB. | ||
| 5106 | 2026.08.17 23:56:01.000361 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 492.48 MiB, peak 503.23 MiB, free memory in arenas 149.10 MiB, will set to 503.87 MiB (RSS), difference: 11.38 MiB | ||
| 5107 | 2026.08.17 23:56:07.918316 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 71979 | ||
| 5108 | 2026.08.17 23:56:07.920153 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5109 | 2026.08.17 23:56:07.920176 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5110 | 2026.08.17 23:56:07.920894 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_31_31_0 (state Temporary) | ||
| 5111 | 2026.08.17 23:56:07.921091 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_31_31_0 to 202608_32_32_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5112 | 2026.08.17 23:56:07.921208 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 71979 | ||
| 5113 | 2026.08.17 23:56:08.284292 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 233 | ||
| 5114 | 2026.08.17 23:56:08.292753 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5115 | 2026.08.17 23:56:08.292770 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5116 | 2026.08.17 23:56:08.296241 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_31_31_0 (state Temporary) | ||
| 5117 | 2026.08.17 23:56:08.298842 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_31_31_0 to 202608_32_32_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5118 | 2026.08.17 23:56:08.299607 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 233 | ||
| 5119 | 2026.08.17 23:56:12.000270 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 504.43 MiB, peak 512.59 MiB, free memory in arenas 130.99 MiB, will set to 497.89 MiB (RSS), difference: -6.55 MiB | ||
| 5120 | 2026.08.17 23:56:13.000250 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 497.89 MiB, peak 512.59 MiB, free memory in arenas 130.99 MiB, will set to 492.98 MiB (RSS), difference: -4.91 MiB | ||
| 5121 | 2026.08.17 23:56:15.327294 [ 432160 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5122 | 2026.08.17 23:56:15.327325 [ 432160 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5123 | 2026.08.17 23:56:15.421317 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 74555 | ||
| 5124 | 2026.08.17 23:56:15.423261 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5125 | 2026.08.17 23:56:15.423285 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5126 | 2026.08.17 23:56:15.424006 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_32_32_0 (state Temporary) | ||
| 5127 | 2026.08.17 23:56:15.424211 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_32_32_0 to 202608_33_33_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5128 | 2026.08.17 23:56:15.424332 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 74555 | ||
| 5129 | 2026.08.17 23:56:15.799693 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 241 | ||
| 5130 | 2026.08.17 23:56:15.806597 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5131 | 2026.08.17 23:56:15.806612 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5132 | 2026.08.17 23:56:15.810035 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_32_32_0 (state Temporary) | ||
| 5133 | 2026.08.17 23:56:15.812482 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_32_32_0 to 202608_33_33_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5134 | 2026.08.17 23:56:15.813267 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 241 | ||
| 5135 | 2026.08.17 23:56:16.000804 [ 432270 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.00012855 sec. | ||
| 5136 | 2026.08.17 23:56:22.924429 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 76809 | ||
| 5137 | 2026.08.17 23:56:22.925929 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5138 | 2026.08.17 23:56:22.925947 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5139 | 2026.08.17 23:56:22.926575 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_33_33_0 (state Temporary) | ||
| 5140 | 2026.08.17 23:56:22.926752 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_33_33_0 to 202608_34_34_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5141 | 2026.08.17 23:56:22.926856 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 76809 | ||
| 5142 | 2026.08.17 23:56:23.313353 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 248 | ||
| 5143 | 2026.08.17 23:56:23.320648 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5144 | 2026.08.17 23:56:23.320664 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5145 | 2026.08.17 23:56:23.324162 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_33_33_0 (state Temporary) | ||
| 5146 | 2026.08.17 23:56:23.326661 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_33_33_0 to 202608_34_34_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5147 | 2026.08.17 23:56:23.327454 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 248 | ||
| 5148 | 2026.08.17 23:56:29.000228 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 496.25 MiB, peak 512.59 MiB, free memory in arenas 118.07 MiB, will set to 494.37 MiB (RSS), difference: -1.88 MiB | ||
| 5149 | 2026.08.17 23:56:30.327420 [ 432097 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5150 | 2026.08.17 23:56:30.327450 [ 432097 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5151 | 2026.08.17 23:56:30.426948 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 79385 | ||
| 5152 | 2026.08.17 23:56:30.428423 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5153 | 2026.08.17 23:56:30.428442 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5154 | 2026.08.17 23:56:30.429016 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_34_34_0 (state Temporary) | ||
| 5155 | 2026.08.17 23:56:30.429176 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_34_34_0 to 202608_35_35_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5156 | 2026.08.17 23:56:30.429276 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 79385 | ||
| 5157 | 2026.08.17 23:56:30.827540 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 256 | ||
| 5158 | 2026.08.17 23:56:30.835679 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5159 | 2026.08.17 23:56:30.835696 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.65 GiB. | ||
| 5160 | 2026.08.17 23:56:30.839160 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_34_34_0 (state Temporary) | ||
| 5161 | 2026.08.17 23:56:30.841729 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_34_34_0 to 202608_35_35_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5162 | 2026.08.17 23:56:30.842500 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 256 | ||
| 5163 | 2026.08.17 23:56:31.000261 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 495.10 MiB, peak 512.59 MiB, free memory in arenas 118.30 MiB, will set to 493.57 MiB (RSS), difference: -1.52 MiB | ||
| 5164 | 2026.08.17 23:56:32.058901 [ 432104 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73361090560, merge_with_ttl_allowedtrue) | ||
| 5165 | 2026.08.17 23:56:32.058927 [ 432104 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 5166 | 2026.08.17 23:56:37.929381 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2256 entries to flush up to offset 81641 | ||
| 5167 | 2026.08.17 23:56:37.930977 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5168 | 2026.08.17 23:56:37.930997 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5169 | 2026.08.17 23:56:37.931556 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_35_35_0 (state Temporary) | ||
| 5170 | 2026.08.17 23:56:37.931762 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_35_35_0 to 202608_36_36_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5171 | 2026.08.17 23:56:37.931882 [ 432026 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_31_6 to 202608_36_36_0 | ||
| 5172 | 2026.08.17 23:56:37.931886 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 81641 | ||
| 5173 | 2026.08.17 23:56:37.931908 [ 432026 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5174 | 2026.08.17 23:56:37.931916 [ 432026 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5175 | 2026.08.17 23:56:37.932055 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_31_6 to 202608_36_36_0 into Compact with storage Full | ||
| 5176 | 2026.08.17 23:56:37.932130 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5177 | 2026.08.17 23:56:37.932149 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 10 marks from part 202608_1_31_6, total 72085 rows starting from the beginning of the part | ||
| 5178 | 2026.08.17 23:56:37.932197 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_32_32_0, total 2254 rows starting from the beginning of the part | ||
| 5179 | 2026.08.17 23:56:37.932215 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_33_33_0, total 2576 rows starting from the beginning of the part | ||
| 5180 | 2026.08.17 23:56:37.932235 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_34_34_0, total 2254 rows starting from the beginning of the part | ||
| 5181 | 2026.08.17 23:56:37.932250 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_35_35_0, total 2576 rows starting from the beginning of the part | ||
| 5182 | 2026.08.17 23:56:37.932293 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_36_36_0, total 2256 rows starting from the beginning of the part | ||
| 5183 | 2026.08.17 23:56:37.939109 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 84001 rows, containing 4 columns (4 merged, 0 gathered) in 0.007097606 sec., 11835117.36210773 rows/sec., 170.90 MiB/sec. | ||
| 5184 | 2026.08.17 23:56:37.939252 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Trace> MergedBlockOutputStream: filled checksums 202608_1_36_7 (state Temporary) | ||
| 5185 | 2026.08.17 23:56:37.939389 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_36_7 to 202608_1_36_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5186 | 2026.08.17 23:56:37.939415 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_36_7} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_31_6, 202608_36_36_0] -> 202608_1_36_7 | ||
| 5187 | 2026.08.17 23:56:37.939457 [ 432207 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 5188 | 2026.08.17 23:56:38.342583 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 263 | ||
| 5189 | 2026.08.17 23:56:38.349653 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5190 | 2026.08.17 23:56:38.349669 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5191 | 2026.08.17 23:56:38.353089 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_35_35_0 (state Temporary) | ||
| 5192 | 2026.08.17 23:56:38.355646 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_35_35_0 to 202608_36_36_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5193 | 2026.08.17 23:56:38.355785 [ 432017 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_31_6 to 202608_36_36_0 | ||
| 5194 | 2026.08.17 23:56:38.355801 [ 432017 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5195 | 2026.08.17 23:56:38.355805 [ 432017 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5196 | 2026.08.17 23:56:38.355944 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_31_6 to 202608_36_36_0 into Compact with storage Full | ||
| 5197 | 2026.08.17 23:56:38.356426 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 263 | ||
| 5198 | 2026.08.17 23:56:38.358743 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5199 | 2026.08.17 23:56:38.359117 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_31_6, total 234 rows starting from the beginning of the part | ||
| 5200 | 2026.08.17 23:56:38.361007 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_32_32_0, total 7 rows starting from the beginning of the part | ||
| 5201 | 2026.08.17 23:56:38.362850 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_33_33_0, total 8 rows starting from the beginning of the part | ||
| 5202 | 2026.08.17 23:56:38.364768 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_34_34_0, total 7 rows starting from the beginning of the part | ||
| 5203 | 2026.08.17 23:56:38.369510 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_35_35_0, total 8 rows starting from the beginning of the part | ||
| 5204 | 2026.08.17 23:56:38.371382 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_36_36_0, total 7 rows starting from the beginning of the part | ||
| 5205 | 2026.08.17 23:56:38.397862 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 271 rows, containing 679 columns (679 merged, 0 gathered) in 0.04197551 sec., 6456.145500078498 rows/sec., 4.85 MiB/sec. | ||
| 5206 | 2026.08.17 23:56:38.398622 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Trace> MergedBlockOutputStream: filled checksums 202608_1_36_7 (state Temporary) | ||
| 5207 | 2026.08.17 23:56:38.401953 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_36_7 to 202608_1_36_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5208 | 2026.08.17 23:56:38.401986 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_36_7} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_31_6, 202608_36_36_0] -> 202608_1_36_7 | ||
| 5209 | 2026.08.17 23:56:38.402378 [ 432199 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.85 MiB. | ||
| 5210 | 2026.08.17 23:56:39.000220 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 494.33 MiB, peak 512.59 MiB, free memory in arenas 128.80 MiB, will set to 505.73 MiB (RSS), difference: 11.41 MiB | ||
| 5211 | 2026.08.17 23:56:45.327560 [ 432083 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5212 | 2026.08.17 23:56:45.327589 [ 432083 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5213 | 2026.08.17 23:56:45.432011 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 84217 | ||
| 5214 | 2026.08.17 23:56:45.434034 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5215 | 2026.08.17 23:56:45.434056 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5216 | 2026.08.17 23:56:45.434772 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_36_36_0 (state Temporary) | ||
| 5217 | 2026.08.17 23:56:45.434968 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_36_36_0 to 202608_37_37_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5218 | 2026.08.17 23:56:45.435083 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 84217 | ||
| 5219 | 2026.08.17 23:56:45.856511 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 271 | ||
| 5220 | 2026.08.17 23:56:45.864150 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5221 | 2026.08.17 23:56:45.864165 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5222 | 2026.08.17 23:56:45.867627 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_36_36_0 (state Temporary) | ||
| 5223 | 2026.08.17 23:56:45.870212 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_36_36_0 to 202608_37_37_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5224 | 2026.08.17 23:56:45.870990 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 271 | ||
| 5225 | 2026.08.17 23:56:49.478731 [ 432149 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 5226 | 2026.08.17 23:56:49.478750 [ 432145 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73359937536, merge_with_ttl_allowedtrue) | ||
| 5227 | 2026.08.17 23:56:49.478781 [ 432145 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 5228 | 2026.08.17 23:56:52.935179 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 86471 | ||
| 5229 | 2026.08.17 23:56:52.936448 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5230 | 2026.08.17 23:56:52.936464 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5231 | 2026.08.17 23:56:52.937104 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_37_37_0 (state Temporary) | ||
| 5232 | 2026.08.17 23:56:52.937271 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_37_37_0 to 202608_38_38_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5233 | 2026.08.17 23:56:52.937375 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 86471 | ||
| 5234 | 2026.08.17 23:56:53.371074 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 279 | ||
| 5235 | 2026.08.17 23:56:53.378186 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5236 | 2026.08.17 23:56:53.378203 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5237 | 2026.08.17 23:56:53.381685 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_37_37_0 (state Temporary) | ||
| 5238 | 2026.08.17 23:56:53.384221 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_37_37_0 to 202608_38_38_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5239 | 2026.08.17 23:56:53.385006 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 279 | ||
| 5240 | 2026.08.17 23:56:54.109003 [ 432140 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73359607808, merge_with_ttl_allowedtrue) | ||
| 5241 | 2026.08.17 23:56:54.109018 [ 432140 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 5242 | 2026.08.17 23:56:54.193868 [ 432120 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73359607808, merge_with_ttl_allowedtrue) | ||
| 5243 | 2026.08.17 23:56:54.193886 [ 432120 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 5244 | 2026.08.17 23:57:00.327687 [ 432071 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5245 | 2026.08.17 23:57:00.327715 [ 432071 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5246 | 2026.08.17 23:57:00.437476 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2578 entries to flush up to offset 89049 | ||
| 5247 | 2026.08.17 23:57:00.439423 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5248 | 2026.08.17 23:57:00.439446 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5249 | 2026.08.17 23:57:00.440181 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_38_38_0 (state Temporary) | ||
| 5250 | 2026.08.17 23:57:00.440375 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_38_38_0 to 202608_39_39_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5251 | 2026.08.17 23:57:00.440492 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 89049 | ||
| 5252 | 2026.08.17 23:57:00.885091 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 286 | ||
| 5253 | 2026.08.17 23:57:00.892446 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5254 | 2026.08.17 23:57:00.892461 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5255 | 2026.08.17 23:57:00.895934 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_38_38_0 (state Temporary) | ||
| 5256 | 2026.08.17 23:57:00.898574 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_38_38_0 to 202608_39_39_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5257 | 2026.08.17 23:57:00.899354 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 286 | ||
| 5258 | 2026.08.17 23:57:07.940584 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 91303 | ||
| 5259 | 2026.08.17 23:57:07.941692 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5260 | 2026.08.17 23:57:07.941706 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5261 | 2026.08.17 23:57:07.942190 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_39_39_0 (state Temporary) | ||
| 5262 | 2026.08.17 23:57:07.942339 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_39_39_0 to 202608_40_40_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5263 | 2026.08.17 23:57:07.942428 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 91303 | ||
| 5264 | 2026.08.17 23:57:08.399437 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 294 | ||
| 5265 | 2026.08.17 23:57:08.406446 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5266 | 2026.08.17 23:57:08.406463 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5267 | 2026.08.17 23:57:08.409814 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_39_39_0 (state Temporary) | ||
| 5268 | 2026.08.17 23:57:08.412314 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_39_39_0 to 202608_40_40_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5269 | 2026.08.17 23:57:08.413100 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 294 | ||
| 5270 | 2026.08.17 23:57:15.327810 [ 432118 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5271 | 2026.08.17 23:57:15.327840 [ 432118 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5272 | 2026.08.17 23:57:15.442512 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 93879 | ||
| 5273 | 2026.08.17 23:57:15.443539 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5274 | 2026.08.17 23:57:15.443551 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5275 | 2026.08.17 23:57:15.444100 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_40_40_0 (state Temporary) | ||
| 5276 | 2026.08.17 23:57:15.444232 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_40_40_0 to 202608_41_41_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5277 | 2026.08.17 23:57:15.444311 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 93879 | ||
| 5278 | 2026.08.17 23:57:15.444333 [ 432129 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_36_7 to 202608_41_41_0 | ||
| 5279 | 2026.08.17 23:57:15.444356 [ 432129 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5280 | 2026.08.17 23:57:15.444362 [ 432129 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5281 | 2026.08.17 23:57:15.444519 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_36_7 to 202608_41_41_0 into Compact with storage Full | ||
| 5282 | 2026.08.17 23:57:15.444584 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5283 | 2026.08.17 23:57:15.444603 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 12 marks from part 202608_1_36_7, total 84001 rows starting from the beginning of the part | ||
| 5284 | 2026.08.17 23:57:15.444651 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_37_37_0, total 2576 rows starting from the beginning of the part | ||
| 5285 | 2026.08.17 23:57:15.444667 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_38_38_0, total 2254 rows starting from the beginning of the part | ||
| 5286 | 2026.08.17 23:57:15.444685 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_39_39_0, total 2578 rows starting from the beginning of the part | ||
| 5287 | 2026.08.17 23:57:15.444704 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_40_40_0, total 2254 rows starting from the beginning of the part | ||
| 5288 | 2026.08.17 23:57:15.444717 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_41_41_0, total 2576 rows starting from the beginning of the part | ||
| 5289 | 2026.08.17 23:57:15.452222 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 96239 rows, containing 4 columns (4 merged, 0 gathered) in 0.007773666 sec., 12380130.558735093 rows/sec., 178.60 MiB/sec. | ||
| 5290 | 2026.08.17 23:57:15.452482 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Trace> MergedBlockOutputStream: filled checksums 202608_1_41_8 (state Temporary) | ||
| 5291 | 2026.08.17 23:57:15.452695 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_41_8 to 202608_1_41_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5292 | 2026.08.17 23:57:15.452719 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_41_8} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_36_7, 202608_41_41_0] -> 202608_1_41_8 | ||
| 5293 | 2026.08.17 23:57:15.452763 [ 432227 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.78 MiB. | ||
| 5294 | 2026.08.17 23:57:15.913857 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 301 | ||
| 5295 | 2026.08.17 23:57:15.920801 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5296 | 2026.08.17 23:57:15.920815 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5297 | 2026.08.17 23:57:15.924200 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_40_40_0 (state Temporary) | ||
| 5298 | 2026.08.17 23:57:15.926755 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_40_40_0 to 202608_41_41_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5299 | 2026.08.17 23:57:15.926894 [ 432133 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_36_7 to 202608_41_41_0 | ||
| 5300 | 2026.08.17 23:57:15.926912 [ 432133 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5301 | 2026.08.17 23:57:15.926916 [ 432133 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5302 | 2026.08.17 23:57:15.927046 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_36_7 to 202608_41_41_0 into Compact with storage Full | ||
| 5303 | 2026.08.17 23:57:15.927529 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 301 | ||
| 5304 | 2026.08.17 23:57:15.929811 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5305 | 2026.08.17 23:57:15.930188 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_36_7, total 271 rows starting from the beginning of the part | ||
| 5306 | 2026.08.17 23:57:15.932059 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_37_37_0, total 8 rows starting from the beginning of the part | ||
| 5307 | 2026.08.17 23:57:15.933877 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_38_38_0, total 8 rows starting from the beginning of the part | ||
| 5308 | 2026.08.17 23:57:15.935767 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_39_39_0, total 7 rows starting from the beginning of the part | ||
| 5309 | 2026.08.17 23:57:15.937584 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_40_40_0, total 8 rows starting from the beginning of the part | ||
| 5310 | 2026.08.17 23:57:15.939443 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_41_41_0, total 7 rows starting from the beginning of the part | ||
| 5311 | 2026.08.17 23:57:15.966707 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 309 rows, containing 679 columns (679 merged, 0 gathered) in 0.039711369 sec., 7781.1470060374895 rows/sec., 5.42 MiB/sec. | ||
| 5312 | 2026.08.17 23:57:15.967524 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Trace> MergedBlockOutputStream: filled checksums 202608_1_41_8 (state Temporary) | ||
| 5313 | 2026.08.17 23:57:15.970810 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_41_8 to 202608_1_41_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5314 | 2026.08.17 23:57:15.970844 [ 432212 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_41_8} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_36_7, 202608_41_41_0] -> 202608_1_41_8 | ||
| 5315 | 2026.08.17 23:57:15.971284 [ 432212 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.65 MiB. | ||
| 5316 | 2026.08.17 23:57:16.000188 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 511.03 MiB, peak 524.90 MiB, free memory in arenas 133.71 MiB, will set to 521.78 MiB (RSS), difference: 10.76 MiB | ||
| 5317 | 2026.08.17 23:57:16.000338 [ 432270 ] {} <Trace> AsynchronousMetrics: Scanning /sys/block | ||
| 5318 | 2026.08.17 23:57:22.944482 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 96133 | ||
| 5319 | 2026.08.17 23:57:22.945470 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5320 | 2026.08.17 23:57:22.945485 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5321 | 2026.08.17 23:57:22.945952 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_41_41_0 (state Temporary) | ||
| 5322 | 2026.08.17 23:57:22.946080 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_41_41_0 to 202608_42_42_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5323 | 2026.08.17 23:57:22.946158 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 96133 | ||
| 5324 | 2026.08.17 23:57:23.427604 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 309 | ||
| 5325 | 2026.08.17 23:57:23.435992 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5326 | 2026.08.17 23:57:23.436012 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5327 | 2026.08.17 23:57:23.439442 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_41_41_0 (state Temporary) | ||
| 5328 | 2026.08.17 23:57:23.442061 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_41_41_0 to 202608_42_42_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5329 | 2026.08.17 23:57:23.442830 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 309 | ||
| 5330 | 2026.08.17 23:57:30.327937 [ 432157 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5331 | 2026.08.17 23:57:30.327972 [ 432157 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5332 | 2026.08.17 23:57:30.446236 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 98709 | ||
| 5333 | 2026.08.17 23:57:30.447387 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5334 | 2026.08.17 23:57:30.447403 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5335 | 2026.08.17 23:57:30.447872 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_42_42_0 (state Temporary) | ||
| 5336 | 2026.08.17 23:57:30.448009 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_42_42_0 to 202608_43_43_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5337 | 2026.08.17 23:57:30.448092 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 98709 | ||
| 5338 | 2026.08.17 23:57:30.942916 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 316 | ||
| 5339 | 2026.08.17 23:57:30.951357 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5340 | 2026.08.17 23:57:30.951375 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5341 | 2026.08.17 23:57:30.954867 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_42_42_0 (state Temporary) | ||
| 5342 | 2026.08.17 23:57:30.957508 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_42_42_0 to 202608_43_43_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5343 | 2026.08.17 23:57:30.958285 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 316 | ||
| 5344 | 2026.08.17 23:57:32.059038 [ 432048 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73357410304, merge_with_ttl_allowedtrue) | ||
| 5345 | 2026.08.17 23:57:32.059058 [ 432048 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 5346 | 2026.08.17 23:57:37.948231 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 100963 | ||
| 5347 | 2026.08.17 23:57:37.949191 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5348 | 2026.08.17 23:57:37.949204 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5349 | 2026.08.17 23:57:37.949666 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_43_43_0 (state Temporary) | ||
| 5350 | 2026.08.17 23:57:37.949794 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_43_43_0 to 202608_44_44_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5351 | 2026.08.17 23:57:37.949878 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 100963 | ||
| 5352 | 2026.08.17 23:57:38.458365 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 324 | ||
| 5353 | 2026.08.17 23:57:38.466925 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5354 | 2026.08.17 23:57:38.466940 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5355 | 2026.08.17 23:57:38.470371 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_43_43_0 (state Temporary) | ||
| 5356 | 2026.08.17 23:57:38.472980 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_43_43_0 to 202608_44_44_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5357 | 2026.08.17 23:57:38.473750 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 324 | ||
| 5358 | 2026.08.17 23:57:43.000180 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 524.04 MiB, peak 531.98 MiB, free memory in arenas 129.77 MiB, will set to 520.46 MiB (RSS), difference: -3.58 MiB | ||
| 5359 | 2026.08.17 23:57:45.328067 [ 432072 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5360 | 2026.08.17 23:57:45.328091 [ 432072 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5361 | 2026.08.17 23:57:45.449964 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 103539 | ||
| 5362 | 2026.08.17 23:57:45.451027 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5363 | 2026.08.17 23:57:45.451039 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5364 | 2026.08.17 23:57:45.451407 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_44_44_0 (state Temporary) | ||
| 5365 | 2026.08.17 23:57:45.451534 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_44_44_0 to 202608_45_45_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5366 | 2026.08.17 23:57:45.451619 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 103539 | ||
| 5367 | 2026.08.17 23:57:45.973835 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 331 | ||
| 5368 | 2026.08.17 23:57:45.981500 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5369 | 2026.08.17 23:57:45.981515 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5370 | 2026.08.17 23:57:45.984928 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_44_44_0 (state Temporary) | ||
| 5371 | 2026.08.17 23:57:45.987452 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_44_44_0 to 202608_45_45_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5372 | 2026.08.17 23:57:45.988222 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 331 | ||
| 5373 | 2026.08.17 23:57:49.478818 [ 432064 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 5374 | 2026.08.17 23:57:49.478844 [ 432064 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73356697600, merge_with_ttl_allowedtrue) | ||
| 5375 | 2026.08.17 23:57:49.478849 [ 432064 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 5376 | 2026.08.17 23:57:49.509356 [ 432018 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeCleanupThread): Checking 5 blocks (1 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/fields_i64_local/blocks | ||
| 5377 | 2026.08.17 23:57:49.509552 [ 432018 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 300133ms, ratio: 0, points per minute: 0) | ||
| 5378 | 2026.08.17 23:57:51.790221 [ 432133 ] {} <Trace> oximeter.version (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 302307ms, ratio: 0, points per minute: 0) | ||
| 5379 | 2026.08.17 23:57:52.797180 [ 432031 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 303314ms, ratio: 0, points per minute: 0) | ||
| 5380 | 2026.08.17 23:57:52.951702 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 105793 | ||
| 5381 | 2026.08.17 23:57:52.952658 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5382 | 2026.08.17 23:57:52.952670 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5383 | 2026.08.17 23:57:52.953007 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_45_45_0 (state Temporary) | ||
| 5384 | 2026.08.17 23:57:52.953140 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_45_45_0 to 202608_46_46_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5385 | 2026.08.17 23:57:52.953223 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 105793 | ||
| 5386 | 2026.08.17 23:57:52.953238 [ 432121 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_41_8 to 202608_46_46_0 | ||
| 5387 | 2026.08.17 23:57:52.953270 [ 432121 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5388 | 2026.08.17 23:57:52.953276 [ 432121 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5389 | 2026.08.17 23:57:52.953416 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_41_8 to 202608_46_46_0 into Compact with storage Full | ||
| 5390 | 2026.08.17 23:57:52.953466 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5391 | 2026.08.17 23:57:52.953480 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 13 marks from part 202608_1_41_8, total 96239 rows starting from the beginning of the part | ||
| 5392 | 2026.08.17 23:57:52.953522 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_42_42_0, total 2254 rows starting from the beginning of the part | ||
| 5393 | 2026.08.17 23:57:52.953537 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_43_43_0, total 2576 rows starting from the beginning of the part | ||
| 5394 | 2026.08.17 23:57:52.953550 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_44_44_0, total 2254 rows starting from the beginning of the part | ||
| 5395 | 2026.08.17 23:57:52.953567 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_45_45_0, total 2576 rows starting from the beginning of the part | ||
| 5396 | 2026.08.17 23:57:52.953581 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_46_46_0, total 2254 rows starting from the beginning of the part | ||
| 5397 | 2026.08.17 23:57:52.961773 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 108153 rows, containing 4 columns (4 merged, 0 gathered) in 0.008413726 sec., 12854352.518729514 rows/sec., 185.23 MiB/sec. | ||
| 5398 | 2026.08.17 23:57:52.962062 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Trace> MergedBlockOutputStream: filled checksums 202608_1_46_9 (state Temporary) | ||
| 5399 | 2026.08.17 23:57:52.962244 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_46_9 to 202608_1_46_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5400 | 2026.08.17 23:57:52.962273 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_46_9} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_41_8, 202608_46_46_0] -> 202608_1_46_9 | ||
| 5401 | 2026.08.17 23:57:52.962337 [ 432207 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.78 MiB. | ||
| 5402 | 2026.08.17 23:57:53.000202 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 521.50 MiB, peak 531.98 MiB, free memory in arenas 137.77 MiB, will set to 525.11 MiB (RSS), difference: 3.62 MiB | ||
| 5403 | 2026.08.17 23:57:53.488306 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 339 | ||
| 5404 | 2026.08.17 23:57:53.496708 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5405 | 2026.08.17 23:57:53.496724 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5406 | 2026.08.17 23:57:53.500164 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_45_45_0 (state Temporary) | ||
| 5407 | 2026.08.17 23:57:53.502778 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_45_45_0 to 202608_46_46_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5408 | 2026.08.17 23:57:53.502927 [ 432126 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_41_8 to 202608_46_46_0 | ||
| 5409 | 2026.08.17 23:57:53.502949 [ 432126 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5410 | 2026.08.17 23:57:53.502954 [ 432126 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5411 | 2026.08.17 23:57:53.503031 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_41_8 to 202608_46_46_0 into Compact with storage Full | ||
| 5412 | 2026.08.17 23:57:53.503612 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 339 | ||
| 5413 | 2026.08.17 23:57:53.506214 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5414 | 2026.08.17 23:57:53.506558 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_41_8, total 309 rows starting from the beginning of the part | ||
| 5415 | 2026.08.17 23:57:53.508379 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_42_42_0, total 8 rows starting from the beginning of the part | ||
| 5416 | 2026.08.17 23:57:53.510179 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_43_43_0, total 7 rows starting from the beginning of the part | ||
| 5417 | 2026.08.17 23:57:53.512062 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_44_44_0, total 8 rows starting from the beginning of the part | ||
| 5418 | 2026.08.17 23:57:53.513887 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_45_45_0, total 7 rows starting from the beginning of the part | ||
| 5419 | 2026.08.17 23:57:53.515710 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_46_46_0, total 8 rows starting from the beginning of the part | ||
| 5420 | 2026.08.17 23:57:53.543505 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 347 rows, containing 679 columns (679 merged, 0 gathered) in 0.040507379 sec., 8566.340468486 rows/sec., 5.81 MiB/sec. | ||
| 5421 | 2026.08.17 23:57:53.544305 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Trace> MergedBlockOutputStream: filled checksums 202608_1_46_9 (state Temporary) | ||
| 5422 | 2026.08.17 23:57:53.547628 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_46_9 to 202608_1_46_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5423 | 2026.08.17 23:57:53.547669 [ 432215 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_46_9} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_41_8, 202608_46_46_0] -> 202608_1_46_9 | ||
| 5424 | 2026.08.17 23:57:53.548708 [ 432215 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.63 MiB. | ||
| 5425 | 2026.08.17 23:57:54.000374 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 526.58 MiB, peak 540.45 MiB, free memory in arenas 142.07 MiB, will set to 537.44 MiB (RSS), difference: 10.86 MiB | ||
| 5426 | 2026.08.17 23:57:54.109136 [ 432122 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73356064768, merge_with_ttl_allowedtrue) | ||
| 5427 | 2026.08.17 23:57:54.109152 [ 432122 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 5428 | 2026.08.17 23:57:54.193999 [ 432139 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73356064768, merge_with_ttl_allowedtrue) | ||
| 5429 | 2026.08.17 23:57:54.194015 [ 432139 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 5430 | 2026.08.17 23:57:58.264379 [ 432146 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeCleanupThread): Checking 4 blocks (1 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/measurements_cumulativef64_local/blocks | ||
| 5431 | 2026.08.17 23:57:58.264566 [ 432146 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 308781ms, ratio: 0, points per minute: 0) | ||
| 5432 | 2026.08.17 23:57:59.147503 [ 432149 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeCleanupThread): Checking 5 blocks (1 are not cached) to clear old ones from ZooKeeper., path is /clickhouse/tables/1/fields_uuid_local/blocks | ||
| 5433 | 2026.08.17 23:57:59.147651 [ 432149 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 309771ms, ratio: 0, points per minute: 0) | ||
| 5434 | 2026.08.17 23:58:00.328184 [ 432160 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5435 | 2026.08.17 23:58:00.328207 [ 432160 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5436 | 2026.08.17 23:58:00.453311 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 108369 | ||
| 5437 | 2026.08.17 23:58:00.454374 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5438 | 2026.08.17 23:58:00.454385 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5439 | 2026.08.17 23:58:00.454723 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_46_46_0 (state Temporary) | ||
| 5440 | 2026.08.17 23:58:00.454854 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_46_46_0 to 202608_47_47_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5441 | 2026.08.17 23:58:00.454939 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 108369 | ||
| 5442 | 2026.08.17 23:58:01.003701 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 346 | ||
| 5443 | 2026.08.17 23:58:01.012209 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5444 | 2026.08.17 23:58:01.012225 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.64 GiB. | ||
| 5445 | 2026.08.17 23:58:01.015693 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_46_46_0 (state Temporary) | ||
| 5446 | 2026.08.17 23:58:01.018245 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_46_46_0 to 202608_47_47_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5447 | 2026.08.17 23:58:01.019013 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 346 | ||
| 5448 | 2026.08.17 23:58:07.955028 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 110623 | ||
| 5449 | 2026.08.17 23:58:07.956117 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5450 | 2026.08.17 23:58:07.956132 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5451 | 2026.08.17 23:58:07.956614 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_47_47_0 (state Temporary) | ||
| 5452 | 2026.08.17 23:58:07.956746 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_47_47_0 to 202608_48_48_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5453 | 2026.08.17 23:58:07.956831 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 110623 | ||
| 5454 | 2026.08.17 23:58:08.519094 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 354 | ||
| 5455 | 2026.08.17 23:58:08.526133 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5456 | 2026.08.17 23:58:08.526148 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5457 | 2026.08.17 23:58:08.529523 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_47_47_0 (state Temporary) | ||
| 5458 | 2026.08.17 23:58:08.531960 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_47_47_0 to 202608_48_48_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5459 | 2026.08.17 23:58:08.532738 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 354 | ||
| 5460 | 2026.08.17 23:58:15.328300 [ 432022 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5461 | 2026.08.17 23:58:15.328322 [ 432022 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5462 | 2026.08.17 23:58:15.456912 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 113199 | ||
| 5463 | 2026.08.17 23:58:15.458068 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5464 | 2026.08.17 23:58:15.458083 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5465 | 2026.08.17 23:58:15.458562 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_48_48_0 (state Temporary) | ||
| 5466 | 2026.08.17 23:58:15.458696 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_48_48_0 to 202608_49_49_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5467 | 2026.08.17 23:58:15.458791 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 113199 | ||
| 5468 | 2026.08.17 23:58:16.000582 [ 432270 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 7.513e-05 sec. | ||
| 5469 | 2026.08.17 23:58:16.032823 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 361 | ||
| 5470 | 2026.08.17 23:58:16.040361 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5471 | 2026.08.17 23:58:16.040377 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5472 | 2026.08.17 23:58:16.043786 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_48_48_0 (state Temporary) | ||
| 5473 | 2026.08.17 23:58:16.046247 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_48_48_0 to 202608_49_49_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5474 | 2026.08.17 23:58:16.047022 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 361 | ||
| 5475 | 2026.08.17 23:58:22.958875 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 115453 | ||
| 5476 | 2026.08.17 23:58:22.959925 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5477 | 2026.08.17 23:58:22.959940 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5478 | 2026.08.17 23:58:22.960413 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_49_49_0 (state Temporary) | ||
| 5479 | 2026.08.17 23:58:22.960541 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_49_49_0 to 202608_50_50_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5480 | 2026.08.17 23:58:22.960625 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 115453 | ||
| 5481 | 2026.08.17 23:58:23.547104 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 369 | ||
| 5482 | 2026.08.17 23:58:23.554996 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5483 | 2026.08.17 23:58:23.555011 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5484 | 2026.08.17 23:58:23.558480 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_49_49_0 (state Temporary) | ||
| 5485 | 2026.08.17 23:58:23.561000 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_49_49_0 to 202608_50_50_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5486 | 2026.08.17 23:58:23.561769 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 369 | ||
| 5487 | 2026.08.17 23:58:30.328414 [ 432076 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5488 | 2026.08.17 23:58:30.328438 [ 432076 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5489 | 2026.08.17 23:58:30.460708 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 118029 | ||
| 5490 | 2026.08.17 23:58:30.461835 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5491 | 2026.08.17 23:58:30.461848 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5492 | 2026.08.17 23:58:30.462317 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_50_50_0 (state Temporary) | ||
| 5493 | 2026.08.17 23:58:30.462447 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_50_50_0 to 202608_51_51_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5494 | 2026.08.17 23:58:30.462530 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 118029 | ||
| 5495 | 2026.08.17 23:58:30.462535 [ 432122 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_46_9 to 202608_51_51_0 | ||
| 5496 | 2026.08.17 23:58:30.462556 [ 432122 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5497 | 2026.08.17 23:58:30.462562 [ 432122 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5498 | 2026.08.17 23:58:30.462650 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_46_9 to 202608_51_51_0 into Compact with storage Full | ||
| 5499 | 2026.08.17 23:58:30.462717 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5500 | 2026.08.17 23:58:30.462735 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 15 marks from part 202608_1_46_9, total 108153 rows starting from the beginning of the part | ||
| 5501 | 2026.08.17 23:58:30.462779 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_47_47_0, total 2576 rows starting from the beginning of the part | ||
| 5502 | 2026.08.17 23:58:30.462797 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_48_48_0, total 2254 rows starting from the beginning of the part | ||
| 5503 | 2026.08.17 23:58:30.462811 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_49_49_0, total 2576 rows starting from the beginning of the part | ||
| 5504 | 2026.08.17 23:58:30.462830 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_50_50_0, total 2254 rows starting from the beginning of the part | ||
| 5505 | 2026.08.17 23:58:30.462846 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_51_51_0, total 2576 rows starting from the beginning of the part | ||
| 5506 | 2026.08.17 23:58:30.471082 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 120389 rows, containing 4 columns (4 merged, 0 gathered) in 0.008470946 sec., 14211990.018588243 rows/sec., 204.68 MiB/sec. | ||
| 5507 | 2026.08.17 23:58:30.471286 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Trace> MergedBlockOutputStream: filled checksums 202608_1_51_10 (state Temporary) | ||
| 5508 | 2026.08.17 23:58:30.471419 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_51_10 to 202608_1_51_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5509 | 2026.08.17 23:58:30.471444 [ 432227 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_51_10} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_46_9, 202608_51_51_0] -> 202608_1_51_10 | ||
| 5510 | 2026.08.17 23:58:30.471490 [ 432227 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 5511 | 2026.08.17 23:58:31.061852 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 376 | ||
| 5512 | 2026.08.17 23:58:31.070349 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5513 | 2026.08.17 23:58:31.070363 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5514 | 2026.08.17 23:58:31.073873 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_50_50_0 (state Temporary) | ||
| 5515 | 2026.08.17 23:58:31.076460 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_50_50_0 to 202608_51_51_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5516 | 2026.08.17 23:58:31.076603 [ 432084 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_46_9 to 202608_51_51_0 | ||
| 5517 | 2026.08.17 23:58:31.076623 [ 432084 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5518 | 2026.08.17 23:58:31.076628 [ 432084 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5519 | 2026.08.17 23:58:31.076702 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_46_9 to 202608_51_51_0 into Compact with storage Full | ||
| 5520 | 2026.08.17 23:58:31.077349 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 376 | ||
| 5521 | 2026.08.17 23:58:31.080556 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5522 | 2026.08.17 23:58:31.080903 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_46_9, total 347 rows starting from the beginning of the part | ||
| 5523 | 2026.08.17 23:58:31.082695 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_47_47_0, total 7 rows starting from the beginning of the part | ||
| 5524 | 2026.08.17 23:58:31.084449 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_48_48_0, total 8 rows starting from the beginning of the part | ||
| 5525 | 2026.08.17 23:58:31.086296 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_49_49_0, total 7 rows starting from the beginning of the part | ||
| 5526 | 2026.08.17 23:58:31.088040 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_50_50_0, total 8 rows starting from the beginning of the part | ||
| 5527 | 2026.08.17 23:58:31.089780 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_51_51_0, total 7 rows starting from the beginning of the part | ||
| 5528 | 2026.08.17 23:58:31.116837 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 384 rows, containing 679 columns (679 merged, 0 gathered) in 0.040163809 sec., 9560.846183687408 rows/sec., 6.30 MiB/sec. | ||
| 5529 | 2026.08.17 23:58:31.117750 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Trace> MergedBlockOutputStream: filled checksums 202608_1_51_10 (state Temporary) | ||
| 5530 | 2026.08.17 23:58:31.121157 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_51_10 to 202608_1_51_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5531 | 2026.08.17 23:58:31.121191 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_51_10} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_46_9, 202608_51_51_0] -> 202608_1_51_10 | ||
| 5532 | 2026.08.17 23:58:31.121606 [ 432224 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.63 MiB. | ||
| 5533 | 2026.08.17 23:58:32.000178 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 542.12 MiB, peak 555.99 MiB, free memory in arenas 151.43 MiB, will set to 545.42 MiB (RSS), difference: 3.30 MiB | ||
| 5534 | 2026.08.17 23:58:32.059167 [ 432105 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73353852928, merge_with_ttl_allowedtrue) | ||
| 5535 | 2026.08.17 23:58:32.059189 [ 432105 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 5536 | 2026.08.17 23:58:37.962702 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 120283 | ||
| 5537 | 2026.08.17 23:58:37.963693 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5538 | 2026.08.17 23:58:37.963706 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5539 | 2026.08.17 23:58:37.964171 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_51_51_0 (state Temporary) | ||
| 5540 | 2026.08.17 23:58:37.964304 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_51_51_0 to 202608_52_52_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5541 | 2026.08.17 23:58:37.964385 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 120283 | ||
| 5542 | 2026.08.17 23:58:38.577433 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 384 | ||
| 5543 | 2026.08.17 23:58:38.584543 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5544 | 2026.08.17 23:58:38.584558 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5545 | 2026.08.17 23:58:38.587954 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_51_51_0 (state Temporary) | ||
| 5546 | 2026.08.17 23:58:38.590414 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_51_51_0 to 202608_52_52_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5547 | 2026.08.17 23:58:38.591209 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 384 | ||
| 5548 | 2026.08.17 23:58:45.328533 [ 432082 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5549 | 2026.08.17 23:58:45.328564 [ 432082 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5550 | 2026.08.17 23:58:45.464469 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 122859 | ||
| 5551 | 2026.08.17 23:58:45.465598 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5552 | 2026.08.17 23:58:45.465611 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5553 | 2026.08.17 23:58:45.466091 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_52_52_0 (state Temporary) | ||
| 5554 | 2026.08.17 23:58:45.466225 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_52_52_0 to 202608_53_53_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5555 | 2026.08.17 23:58:45.466309 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 122859 | ||
| 5556 | 2026.08.17 23:58:46.091294 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 391 | ||
| 5557 | 2026.08.17 23:58:46.098782 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5558 | 2026.08.17 23:58:46.098796 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5559 | 2026.08.17 23:58:46.102179 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_52_52_0 (state Temporary) | ||
| 5560 | 2026.08.17 23:58:46.104681 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_52_52_0 to 202608_53_53_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5561 | 2026.08.17 23:58:46.105468 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 391 | ||
| 5562 | 2026.08.17 23:58:49.478910 [ 432106 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 5563 | 2026.08.17 23:58:49.478921 [ 432102 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73352982528, merge_with_ttl_allowedtrue) | ||
| 5564 | 2026.08.17 23:58:49.478943 [ 432102 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 5565 | 2026.08.17 23:58:52.966394 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 125113 | ||
| 5566 | 2026.08.17 23:58:52.967369 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5567 | 2026.08.17 23:58:52.967381 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5568 | 2026.08.17 23:58:52.967731 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_53_53_0 (state Temporary) | ||
| 5569 | 2026.08.17 23:58:52.967865 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_53_53_0 to 202608_54_54_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5570 | 2026.08.17 23:58:52.967948 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 125113 | ||
| 5571 | 2026.08.17 23:58:53.605552 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 399 | ||
| 5572 | 2026.08.17 23:58:53.613477 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5573 | 2026.08.17 23:58:53.613494 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5574 | 2026.08.17 23:58:53.616959 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_53_53_0 (state Temporary) | ||
| 5575 | 2026.08.17 23:58:53.619553 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_53_53_0 to 202608_54_54_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5576 | 2026.08.17 23:58:53.620321 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 399 | ||
| 5577 | 2026.08.17 23:58:54.109267 [ 432099 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73352650752, merge_with_ttl_allowedtrue) | ||
| 5578 | 2026.08.17 23:58:54.109288 [ 432099 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 5579 | 2026.08.17 23:58:54.194120 [ 432061 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73352650752, merge_with_ttl_allowedtrue) | ||
| 5580 | 2026.08.17 23:58:54.194136 [ 432061 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 5581 | 2026.08.17 23:59:00.328659 [ 432018 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5582 | 2026.08.17 23:59:00.328688 [ 432018 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5583 | 2026.08.17 23:59:00.468022 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 127689 | ||
| 5584 | 2026.08.17 23:59:00.469194 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5585 | 2026.08.17 23:59:00.469207 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5586 | 2026.08.17 23:59:00.469708 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_54_54_0 (state Temporary) | ||
| 5587 | 2026.08.17 23:59:00.469838 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_54_54_0 to 202608_55_55_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5588 | 2026.08.17 23:59:00.469923 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 127689 | ||
| 5589 | 2026.08.17 23:59:01.120405 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 406 | ||
| 5590 | 2026.08.17 23:59:01.128194 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5591 | 2026.08.17 23:59:01.128210 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5592 | 2026.08.17 23:59:01.131661 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_54_54_0 (state Temporary) | ||
| 5593 | 2026.08.17 23:59:01.134232 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_54_54_0 to 202608_55_55_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5594 | 2026.08.17 23:59:01.135009 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 406 | ||
| 5595 | 2026.08.17 23:59:07.970012 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 129943 | ||
| 5596 | 2026.08.17 23:59:07.970997 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5597 | 2026.08.17 23:59:07.971010 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5598 | 2026.08.17 23:59:07.971388 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_55_55_0 (state Temporary) | ||
| 5599 | 2026.08.17 23:59:07.971520 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_55_55_0 to 202608_56_56_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5600 | 2026.08.17 23:59:07.971607 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 129943 | ||
| 5601 | 2026.08.17 23:59:07.971604 [ 432153 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_51_10 to 202608_56_56_0 | ||
| 5602 | 2026.08.17 23:59:07.971625 [ 432153 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5603 | 2026.08.17 23:59:07.971630 [ 432153 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5604 | 2026.08.17 23:59:07.971745 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_51_10 to 202608_56_56_0 into Compact with storage Full | ||
| 5605 | 2026.08.17 23:59:07.971782 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5606 | 2026.08.17 23:59:07.971795 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 16 marks from part 202608_1_51_10, total 120389 rows starting from the beginning of the part | ||
| 5607 | 2026.08.17 23:59:07.971838 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_52_52_0, total 2254 rows starting from the beginning of the part | ||
| 5608 | 2026.08.17 23:59:07.971854 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_53_53_0, total 2576 rows starting from the beginning of the part | ||
| 5609 | 2026.08.17 23:59:07.971870 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_54_54_0, total 2254 rows starting from the beginning of the part | ||
| 5610 | 2026.08.17 23:59:07.971890 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_55_55_0, total 2576 rows starting from the beginning of the part | ||
| 5611 | 2026.08.17 23:59:07.971903 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_56_56_0, total 2254 rows starting from the beginning of the part | ||
| 5612 | 2026.08.17 23:59:07.981414 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 132303 rows, containing 4 columns (4 merged, 0 gathered) in 0.009724427 sec., 13605223.217779309 rows/sec., 195.79 MiB/sec. | ||
| 5613 | 2026.08.17 23:59:07.981545 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Trace> MergedBlockOutputStream: filled checksums 202608_1_56_11 (state Temporary) | ||
| 5614 | 2026.08.17 23:59:07.981664 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_56_11 to 202608_1_56_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5615 | 2026.08.17 23:59:07.981687 [ 432202 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_56_11} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_51_10, 202608_56_56_0] -> 202608_1_56_11 | ||
| 5616 | 2026.08.17 23:59:07.981727 [ 432202 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.79 MiB. | ||
| 5617 | 2026.08.17 23:59:08.000248 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 549.64 MiB, peak 557.80 MiB, free memory in arenas 153.03 MiB, will set to 552.76 MiB (RSS), difference: 3.12 MiB | ||
| 5618 | 2026.08.17 23:59:08.635093 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 414 | ||
| 5619 | 2026.08.17 23:59:08.642599 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5620 | 2026.08.17 23:59:08.642616 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5621 | 2026.08.17 23:59:08.646048 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_55_55_0 (state Temporary) | ||
| 5622 | 2026.08.17 23:59:08.648717 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_55_55_0 to 202608_56_56_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5623 | 2026.08.17 23:59:08.648864 [ 432093 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_51_10 to 202608_56_56_0 | ||
| 5624 | 2026.08.17 23:59:08.648877 [ 432093 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5625 | 2026.08.17 23:59:08.648881 [ 432093 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5626 | 2026.08.17 23:59:08.648964 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_51_10 to 202608_56_56_0 into Compact with storage Full | ||
| 5627 | 2026.08.17 23:59:08.649506 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 414 | ||
| 5628 | 2026.08.17 23:59:08.652241 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5629 | 2026.08.17 23:59:08.652583 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_51_10, total 384 rows starting from the beginning of the part | ||
| 5630 | 2026.08.17 23:59:08.654433 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_52_52_0, total 8 rows starting from the beginning of the part | ||
| 5631 | 2026.08.17 23:59:08.656263 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_53_53_0, total 7 rows starting from the beginning of the part | ||
| 5632 | 2026.08.17 23:59:08.658174 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_54_54_0, total 8 rows starting from the beginning of the part | ||
| 5633 | 2026.08.17 23:59:08.660026 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_55_55_0, total 7 rows starting from the beginning of the part | ||
| 5634 | 2026.08.17 23:59:08.661877 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_56_56_0, total 8 rows starting from the beginning of the part | ||
| 5635 | 2026.08.17 23:59:08.689336 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 422 rows, containing 679 columns (679 merged, 0 gathered) in 0.040406209 sec., 10443.93944504915 rows/sec., 6.75 MiB/sec. | ||
| 5636 | 2026.08.17 23:59:08.690173 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Trace> MergedBlockOutputStream: filled checksums 202608_1_56_11 (state Temporary) | ||
| 5637 | 2026.08.17 23:59:08.693486 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_56_11 to 202608_1_56_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5638 | 2026.08.17 23:59:08.693523 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_56_11} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_51_10, 202608_56_56_0] -> 202608_1_56_11 | ||
| 5639 | 2026.08.17 23:59:08.693917 [ 432226 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.65 MiB. | ||
| 5640 | 2026.08.17 23:59:09.000218 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 554.25 MiB, peak 568.12 MiB, free memory in arenas 165.28 MiB, will set to 563.79 MiB (RSS), difference: 9.54 MiB | ||
| 5641 | 2026.08.17 23:59:15.328784 [ 432159 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5642 | 2026.08.17 23:59:15.328815 [ 432159 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5643 | 2026.08.17 23:59:15.471889 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 132519 | ||
| 5644 | 2026.08.17 23:59:15.472970 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5645 | 2026.08.17 23:59:15.472983 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5646 | 2026.08.17 23:59:15.473412 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_56_56_0 (state Temporary) | ||
| 5647 | 2026.08.17 23:59:15.473548 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_56_56_0 to 202608_57_57_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5648 | 2026.08.17 23:59:15.473634 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 132519 | ||
| 5649 | 2026.08.17 23:59:16.000196 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 563.77 MiB, peak 571.97 MiB, free memory in arenas 164.27 MiB, will set to 561.65 MiB (RSS), difference: -2.11 MiB | ||
| 5650 | 2026.08.17 23:59:16.149588 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 421 | ||
| 5651 | 2026.08.17 23:59:16.157462 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5652 | 2026.08.17 23:59:16.157477 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5653 | 2026.08.17 23:59:16.160935 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_56_56_0 (state Temporary) | ||
| 5654 | 2026.08.17 23:59:16.163606 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_56_56_0 to 202608_57_57_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5655 | 2026.08.17 23:59:16.164389 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 421 | ||
| 5656 | 2026.08.17 23:59:22.973720 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 134773 | ||
| 5657 | 2026.08.17 23:59:22.974789 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5658 | 2026.08.17 23:59:22.974803 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5659 | 2026.08.17 23:59:22.975245 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_57_57_0 (state Temporary) | ||
| 5660 | 2026.08.17 23:59:22.975377 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_57_57_0 to 202608_58_58_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5661 | 2026.08.17 23:59:22.975462 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 134773 | ||
| 5662 | 2026.08.17 23:59:23.664472 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 429 | ||
| 5663 | 2026.08.17 23:59:23.672790 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5664 | 2026.08.17 23:59:23.672808 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5665 | 2026.08.17 23:59:23.676283 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_57_57_0 (state Temporary) | ||
| 5666 | 2026.08.17 23:59:23.678967 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_57_57_0 to 202608_58_58_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5667 | 2026.08.17 23:59:23.679754 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 429 | ||
| 5668 | 2026.08.17 23:59:30.328912 [ 432104 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5669 | 2026.08.17 23:59:30.328940 [ 432104 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5670 | 2026.08.17 23:59:30.475535 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 137349 | ||
| 5671 | 2026.08.17 23:59:30.476678 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5672 | 2026.08.17 23:59:30.476691 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5673 | 2026.08.17 23:59:30.477164 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_58_58_0 (state Temporary) | ||
| 5674 | 2026.08.17 23:59:30.477298 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_58_58_0 to 202608_59_59_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5675 | 2026.08.17 23:59:30.477383 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 137349 | ||
| 5676 | 2026.08.17 23:59:31.179842 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 436 | ||
| 5677 | 2026.08.17 23:59:31.187284 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5678 | 2026.08.17 23:59:31.187301 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5679 | 2026.08.17 23:59:31.190728 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_58_58_0 (state Temporary) | ||
| 5680 | 2026.08.17 23:59:31.193350 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_58_58_0 to 202608_59_59_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5681 | 2026.08.17 23:59:31.194133 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 436 | ||
| 5682 | 2026.08.17 23:59:32.059300 [ 432036 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73350424576, merge_with_ttl_allowedtrue) | ||
| 5683 | 2026.08.17 23:59:32.059319 [ 432036 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 5684 | 2026.08.17 23:59:37.977469 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 139603 | ||
| 5685 | 2026.08.17 23:59:37.978553 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5686 | 2026.08.17 23:59:37.978567 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5687 | 2026.08.17 23:59:37.979050 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_59_59_0 (state Temporary) | ||
| 5688 | 2026.08.17 23:59:37.979195 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_59_59_0 to 202608_60_60_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5689 | 2026.08.17 23:59:37.979281 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 139603 | ||
| 5690 | 2026.08.17 23:59:38.694223 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 444 | ||
| 5691 | 2026.08.17 23:59:38.702288 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5692 | 2026.08.17 23:59:38.702307 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.63 GiB. | ||
| 5693 | 2026.08.17 23:59:38.705788 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_59_59_0 (state Temporary) | ||
| 5694 | 2026.08.17 23:59:38.708421 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_59_59_0 to 202608_60_60_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5695 | 2026.08.17 23:59:38.709202 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 444 | ||
| 5696 | 2026.08.17 23:59:45.329039 [ 432083 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5697 | 2026.08.17 23:59:45.329059 [ 432083 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5698 | 2026.08.17 23:59:45.479435 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 142179 | ||
| 5699 | 2026.08.17 23:59:45.480522 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5700 | 2026.08.17 23:59:45.480534 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5701 | 2026.08.17 23:59:45.481018 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_60_60_0 (state Temporary) | ||
| 5702 | 2026.08.17 23:59:45.481151 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_60_60_0 to 202608_61_61_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5703 | 2026.08.17 23:59:45.481236 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 142179 | ||
| 5704 | 2026.08.17 23:59:45.481238 [ 432144 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_56_11 to 202608_61_61_0 | ||
| 5705 | 2026.08.17 23:59:45.481254 [ 432144 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5706 | 2026.08.17 23:59:45.481262 [ 432144 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5707 | 2026.08.17 23:59:45.481356 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_56_11 to 202608_61_61_0 into Compact with storage Full | ||
| 5708 | 2026.08.17 23:59:45.481432 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5709 | 2026.08.17 23:59:45.481451 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 18 marks from part 202608_1_56_11, total 132303 rows starting from the beginning of the part | ||
| 5710 | 2026.08.17 23:59:45.481493 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_57_57_0, total 2576 rows starting from the beginning of the part | ||
| 5711 | 2026.08.17 23:59:45.481514 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_58_58_0, total 2254 rows starting from the beginning of the part | ||
| 5712 | 2026.08.17 23:59:45.481531 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_59_59_0, total 2576 rows starting from the beginning of the part | ||
| 5713 | 2026.08.17 23:59:45.481547 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_60_60_0, total 2254 rows starting from the beginning of the part | ||
| 5714 | 2026.08.17 23:59:45.481564 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_61_61_0, total 2576 rows starting from the beginning of the part | ||
| 5715 | 2026.08.17 23:59:45.492030 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 144539 rows, containing 4 columns (4 merged, 0 gathered) in 0.010717668 sec., 13486049.390595045 rows/sec., 194.01 MiB/sec. | ||
| 5716 | 2026.08.17 23:59:45.492205 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Trace> MergedBlockOutputStream: filled checksums 202608_1_61_12 (state Temporary) | ||
| 5717 | 2026.08.17 23:59:45.492339 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_61_12 to 202608_1_61_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5718 | 2026.08.17 23:59:45.492366 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_61_12} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_56_11, 202608_61_61_0] -> 202608_1_61_12 | ||
| 5719 | 2026.08.17 23:59:45.492409 [ 432207 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 5720 | 2026.08.17 23:59:46.000186 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 564.71 MiB, peak 572.89 MiB, free memory in arenas 164.87 MiB, will set to 560.34 MiB (RSS), difference: -4.37 MiB | ||
| 5721 | 2026.08.17 23:59:46.209285 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 451 | ||
| 5722 | 2026.08.17 23:59:46.216757 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5723 | 2026.08.17 23:59:46.216774 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5724 | 2026.08.17 23:59:46.220198 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_60_60_0 (state Temporary) | ||
| 5725 | 2026.08.17 23:59:46.222737 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_60_60_0 to 202608_61_61_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5726 | 2026.08.17 23:59:46.222875 [ 432093 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_56_11 to 202608_61_61_0 | ||
| 5727 | 2026.08.17 23:59:46.222888 [ 432093 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5728 | 2026.08.17 23:59:46.222892 [ 432093 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5729 | 2026.08.17 23:59:46.222963 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_56_11 to 202608_61_61_0 into Compact with storage Full | ||
| 5730 | 2026.08.17 23:59:46.223532 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 451 | ||
| 5731 | 2026.08.17 23:59:46.226215 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5732 | 2026.08.17 23:59:46.226524 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_56_11, total 422 rows starting from the beginning of the part | ||
| 5733 | 2026.08.17 23:59:46.228276 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_57_57_0, total 7 rows starting from the beginning of the part | ||
| 5734 | 2026.08.17 23:59:46.229958 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_58_58_0, total 8 rows starting from the beginning of the part | ||
| 5735 | 2026.08.17 23:59:46.231724 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_59_59_0, total 7 rows starting from the beginning of the part | ||
| 5736 | 2026.08.17 23:59:46.233393 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_60_60_0, total 8 rows starting from the beginning of the part | ||
| 5737 | 2026.08.17 23:59:46.235071 [ 432226 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_61_61_0, total 7 rows starting from the beginning of the part | ||
| 5738 | 2026.08.17 23:59:46.261903 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 459 rows, containing 679 columns (679 merged, 0 gathered) in 0.038964458 sec., 11779.966245135503 rows/sec., 7.46 MiB/sec. | ||
| 5739 | 2026.08.17 23:59:46.262891 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Trace> MergedBlockOutputStream: filled checksums 202608_1_61_12 (state Temporary) | ||
| 5740 | 2026.08.17 23:59:46.266531 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_61_12 to 202608_1_61_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5741 | 2026.08.17 23:59:46.266569 [ 432210 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_61_12} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_56_11, 202608_61_61_0] -> 202608_1_61_12 | ||
| 5742 | 2026.08.17 23:59:46.266998 [ 432210 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.63 MiB. | ||
| 5743 | 2026.08.17 23:59:47.000204 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 561.77 MiB, peak 575.67 MiB, free memory in arenas 164.54 MiB, will set to 565.43 MiB (RSS), difference: 3.66 MiB | ||
| 5744 | 2026.08.17 23:59:49.479008 [ 432145 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 5745 | 2026.08.17 23:59:49.479016 [ 432152 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73349400576, merge_with_ttl_allowedtrue) | ||
| 5746 | 2026.08.17 23:59:49.479041 [ 432152 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 5747 | 2026.08.17 23:59:52.981322 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 144433 | ||
| 5748 | 2026.08.17 23:59:52.982317 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5749 | 2026.08.17 23:59:52.982330 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5750 | 2026.08.17 23:59:52.982875 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_61_61_0 (state Temporary) | ||
| 5751 | 2026.08.17 23:59:52.983011 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_61_61_0 to 202608_62_62_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5752 | 2026.08.17 23:59:52.983097 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 144433 | ||
| 5753 | 2026.08.17 23:59:53.723616 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 459 | ||
| 5754 | 2026.08.17 23:59:53.731223 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5755 | 2026.08.17 23:59:53.731240 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5756 | 2026.08.17 23:59:53.734697 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_61_61_0 (state Temporary) | ||
| 5757 | 2026.08.17 23:59:53.737259 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_61_61_0 to 202608_62_62_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5758 | 2026.08.17 23:59:53.738047 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 459 | ||
| 5759 | 2026.08.17 23:59:54.000198 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 566.24 MiB, peak 575.67 MiB, free memory in arenas 164.34 MiB, will set to 567.58 MiB (RSS), difference: 1.34 MiB | ||
| 5760 | 2026.08.17 23:59:54.109408 [ 432132 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73348913152, merge_with_ttl_allowedtrue) | ||
| 5761 | 2026.08.17 23:59:54.109425 [ 432132 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 5762 | 2026.08.17 23:59:54.194239 [ 432135 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73348913152, merge_with_ttl_allowedtrue) | ||
| 5763 | 2026.08.17 23:59:54.194258 [ 432135 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 5764 | 2026.08.18 00:00:00.329152 [ 432057 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5765 | 2026.08.18 00:00:00.329176 [ 432057 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5766 | 2026.08.18 00:00:00.483184 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 147009 | ||
| 5767 | 2026.08.18 00:00:00.484358 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5768 | 2026.08.18 00:00:00.484371 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5769 | 2026.08.18 00:00:00.484855 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_62_62_0 (state Temporary) | ||
| 5770 | 2026.08.18 00:00:00.484987 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_62_62_0 to 202608_63_63_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5771 | 2026.08.18 00:00:00.485077 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 147009 | ||
| 5772 | 2026.08.18 00:00:01.238129 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 466 | ||
| 5773 | 2026.08.18 00:00:01.246007 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5774 | 2026.08.18 00:00:01.246023 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5775 | 2026.08.18 00:00:01.249488 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_62_62_0 (state Temporary) | ||
| 5776 | 2026.08.18 00:00:01.252056 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_62_62_0 to 202608_63_63_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5777 | 2026.08.18 00:00:01.252838 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 466 | ||
| 5778 | 2026.08.18 00:00:02.000361 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 568.06 MiB, peak 576.09 MiB, free memory in arenas 164.30 MiB, will set to 566.93 MiB (RSS), difference: -1.12 MiB | ||
| 5779 | 2026.08.18 00:00:07.985169 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 149263 | ||
| 5780 | 2026.08.18 00:00:07.986283 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5781 | 2026.08.18 00:00:07.986298 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5782 | 2026.08.18 00:00:07.986791 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_63_63_0 (state Temporary) | ||
| 5783 | 2026.08.18 00:00:07.986924 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_63_63_0 to 202608_64_64_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5784 | 2026.08.18 00:00:07.987012 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 149263 | ||
| 5785 | 2026.08.18 00:00:08.752923 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 474 | ||
| 5786 | 2026.08.18 00:00:08.760100 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5787 | 2026.08.18 00:00:08.760114 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5788 | 2026.08.18 00:00:08.763575 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_63_63_0 (state Temporary) | ||
| 5789 | 2026.08.18 00:00:08.766145 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_63_63_0 to 202608_64_64_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5790 | 2026.08.18 00:00:08.766942 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 474 | ||
| 5791 | 2026.08.18 00:00:15.329352 [ 432133 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5792 | 2026.08.18 00:00:15.329378 [ 432133 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5793 | 2026.08.18 00:00:15.487098 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 151839 | ||
| 5794 | 2026.08.18 00:00:15.488261 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5795 | 2026.08.18 00:00:15.488276 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5796 | 2026.08.18 00:00:15.488759 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_64_64_0 (state Temporary) | ||
| 5797 | 2026.08.18 00:00:15.488896 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_64_64_0 to 202608_65_65_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5798 | 2026.08.18 00:00:15.488985 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 151839 | ||
| 5799 | 2026.08.18 00:00:16.000856 [ 432270 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.000127911 sec. | ||
| 5800 | 2026.08.18 00:00:16.267106 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 481 | ||
| 5801 | 2026.08.18 00:00:16.274905 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5802 | 2026.08.18 00:00:16.274919 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5803 | 2026.08.18 00:00:16.278342 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_64_64_0 (state Temporary) | ||
| 5804 | 2026.08.18 00:00:16.280766 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_64_64_0 to 202608_65_65_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5805 | 2026.08.18 00:00:16.281540 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 481 | ||
| 5806 | 2026.08.18 00:00:22.989167 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 154093 | ||
| 5807 | 2026.08.18 00:00:22.990065 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5808 | 2026.08.18 00:00:22.990077 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5809 | 2026.08.18 00:00:22.990403 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_65_65_0 (state Temporary) | ||
| 5810 | 2026.08.18 00:00:22.990526 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_65_65_0 to 202608_66_66_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5811 | 2026.08.18 00:00:22.990601 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 154093 | ||
| 5812 | 2026.08.18 00:00:22.990627 [ 432040 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_61_12 to 202608_66_66_0 | ||
| 5813 | 2026.08.18 00:00:22.990649 [ 432040 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5814 | 2026.08.18 00:00:22.990654 [ 432040 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5815 | 2026.08.18 00:00:22.990745 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_61_12 to 202608_66_66_0 into Compact with storage Full | ||
| 5816 | 2026.08.18 00:00:22.990803 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5817 | 2026.08.18 00:00:22.990817 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 19 marks from part 202608_1_61_12, total 144539 rows starting from the beginning of the part | ||
| 5818 | 2026.08.18 00:00:22.990863 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_62_62_0, total 2254 rows starting from the beginning of the part | ||
| 5819 | 2026.08.18 00:00:22.990880 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_63_63_0, total 2576 rows starting from the beginning of the part | ||
| 5820 | 2026.08.18 00:00:22.990896 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_64_64_0, total 2254 rows starting from the beginning of the part | ||
| 5821 | 2026.08.18 00:00:22.990916 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_65_65_0, total 2576 rows starting from the beginning of the part | ||
| 5822 | 2026.08.18 00:00:22.990931 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_66_66_0, total 2254 rows starting from the beginning of the part | ||
| 5823 | 2026.08.18 00:00:23.000194 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 576.82 MiB, peak 577.54 MiB, free memory in arenas 149.34 MiB, will set to 569.52 MiB (RSS), difference: -7.30 MiB | ||
| 5824 | 2026.08.18 00:00:23.001628 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 156453 rows, containing 4 columns (4 merged, 0 gathered) in 0.010920298 sec., 14326806.832560796 rows/sec., 205.99 MiB/sec. | ||
| 5825 | 2026.08.18 00:00:23.001752 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Trace> MergedBlockOutputStream: filled checksums 202608_1_66_13 (state Temporary) | ||
| 5826 | 2026.08.18 00:00:23.001882 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_66_13 to 202608_1_66_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5827 | 2026.08.18 00:00:23.001907 [ 432215 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_66_13} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_61_12, 202608_66_66_0] -> 202608_1_66_13 | ||
| 5828 | 2026.08.18 00:00:23.001966 [ 432215 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 5829 | 2026.08.18 00:00:23.781701 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 489 | ||
| 5830 | 2026.08.18 00:00:23.790408 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5831 | 2026.08.18 00:00:23.790423 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5832 | 2026.08.18 00:00:23.793891 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_65_65_0 (state Temporary) | ||
| 5833 | 2026.08.18 00:00:23.796534 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_65_65_0 to 202608_66_66_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5834 | 2026.08.18 00:00:23.796677 [ 432116 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_61_12 to 202608_66_66_0 | ||
| 5835 | 2026.08.18 00:00:23.796687 [ 432116 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5836 | 2026.08.18 00:00:23.796691 [ 432116 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5837 | 2026.08.18 00:00:23.796767 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_61_12 to 202608_66_66_0 into Compact with storage Full | ||
| 5838 | 2026.08.18 00:00:23.797437 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 489 | ||
| 5839 | 2026.08.18 00:00:23.800528 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5840 | 2026.08.18 00:00:23.800930 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_61_12, total 459 rows starting from the beginning of the part | ||
| 5841 | 2026.08.18 00:00:23.802891 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_62_62_0, total 8 rows starting from the beginning of the part | ||
| 5842 | 2026.08.18 00:00:23.804766 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_63_63_0, total 7 rows starting from the beginning of the part | ||
| 5843 | 2026.08.18 00:00:23.806691 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_64_64_0, total 8 rows starting from the beginning of the part | ||
| 5844 | 2026.08.18 00:00:23.808554 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_65_65_0, total 7 rows starting from the beginning of the part | ||
| 5845 | 2026.08.18 00:00:23.810404 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_66_66_0, total 8 rows starting from the beginning of the part | ||
| 5846 | 2026.08.18 00:00:23.838218 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 497 rows, containing 679 columns (679 merged, 0 gathered) in 0.041480119 sec., 11981.643543500923 rows/sec., 7.49 MiB/sec. | ||
| 5847 | 2026.08.18 00:00:23.839143 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Trace> MergedBlockOutputStream: filled checksums 202608_1_66_13 (state Temporary) | ||
| 5848 | 2026.08.18 00:00:23.842462 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_66_13 to 202608_1_66_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5849 | 2026.08.18 00:00:23.842501 [ 432197 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_66_13} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_61_12, 202608_66_66_0] -> 202608_1_66_13 | ||
| 5850 | 2026.08.18 00:00:23.842923 [ 432197 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.66 MiB. | ||
| 5851 | 2026.08.18 00:00:24.000447 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 563.45 MiB, peak 577.54 MiB, free memory in arenas 173.42 MiB, will set to 584.57 MiB (RSS), difference: 21.12 MiB | ||
| 5852 | 2026.08.18 00:00:29.000254 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 584.56 MiB, peak 584.57 MiB, free memory in arenas 149.97 MiB, will set to 582.64 MiB (RSS), difference: -1.92 MiB | ||
| 5853 | 2026.08.18 00:00:30.329561 [ 432141 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5854 | 2026.08.18 00:00:30.329592 [ 432141 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5855 | 2026.08.18 00:00:30.490772 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 156669 | ||
| 5856 | 2026.08.18 00:00:30.492081 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5857 | 2026.08.18 00:00:30.492098 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5858 | 2026.08.18 00:00:30.492602 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_66_66_0 (state Temporary) | ||
| 5859 | 2026.08.18 00:00:30.492732 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_66_66_0 to 202608_67_67_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5860 | 2026.08.18 00:00:30.492816 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 156669 | ||
| 5861 | 2026.08.18 00:00:31.297610 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 496 | ||
| 5862 | 2026.08.18 00:00:31.306523 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5863 | 2026.08.18 00:00:31.306537 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5864 | 2026.08.18 00:00:31.309975 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_66_66_0 (state Temporary) | ||
| 5865 | 2026.08.18 00:00:31.312508 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_66_66_0 to 202608_67_67_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5866 | 2026.08.18 00:00:31.313275 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 496 | ||
| 5867 | 2026.08.18 00:00:32.000188 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 582.69 MiB, peak 590.72 MiB, free memory in arenas 160.37 MiB, will set to 572.46 MiB (RSS), difference: -10.23 MiB | ||
| 5868 | 2026.08.18 00:00:32.059433 [ 432048 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346293760, merge_with_ttl_allowedtrue) | ||
| 5869 | 2026.08.18 00:00:32.059443 [ 432048 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 5870 | 2026.08.18 00:00:37.000255 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 572.46 MiB, peak 590.72 MiB, free memory in arenas 160.17 MiB, will set to 564.97 MiB (RSS), difference: -7.49 MiB | ||
| 5871 | 2026.08.18 00:00:37.992992 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 158923 | ||
| 5872 | 2026.08.18 00:00:37.993976 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5873 | 2026.08.18 00:00:37.993988 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5874 | 2026.08.18 00:00:37.994393 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_67_67_0 (state Temporary) | ||
| 5875 | 2026.08.18 00:00:37.994516 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_67_67_0 to 202608_68_68_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5876 | 2026.08.18 00:00:37.994590 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 158923 | ||
| 5877 | 2026.08.18 00:00:38.000161 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 564.97 MiB, peak 590.72 MiB, free memory in arenas 160.52 MiB, will set to 563.41 MiB (RSS), difference: -1.56 MiB | ||
| 5878 | 2026.08.18 00:00:38.813360 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 504 | ||
| 5879 | 2026.08.18 00:00:38.822132 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5880 | 2026.08.18 00:00:38.822148 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5881 | 2026.08.18 00:00:38.825638 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_67_67_0 (state Temporary) | ||
| 5882 | 2026.08.18 00:00:38.828246 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_67_67_0 to 202608_68_68_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5883 | 2026.08.18 00:00:38.829018 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 504 | ||
| 5884 | 2026.08.18 00:00:45.329776 [ 432036 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5885 | 2026.08.18 00:00:45.329794 [ 432036 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5886 | 2026.08.18 00:00:45.494755 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 161499 | ||
| 5887 | 2026.08.18 00:00:45.495777 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5888 | 2026.08.18 00:00:45.495788 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5889 | 2026.08.18 00:00:45.496104 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_68_68_0 (state Temporary) | ||
| 5890 | 2026.08.18 00:00:45.496230 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_68_68_0 to 202608_69_69_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5891 | 2026.08.18 00:00:45.496306 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 161499 | ||
| 5892 | 2026.08.18 00:00:46.329187 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 511 | ||
| 5893 | 2026.08.18 00:00:46.336323 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5894 | 2026.08.18 00:00:46.336338 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5895 | 2026.08.18 00:00:46.339791 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_68_68_0 (state Temporary) | ||
| 5896 | 2026.08.18 00:00:46.342238 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_68_68_0 to 202608_69_69_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5897 | 2026.08.18 00:00:46.343040 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 511 | ||
| 5898 | 2026.08.18 00:00:47.000245 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 564.91 MiB, peak 590.72 MiB, free memory in arenas 161.32 MiB, will set to 566.09 MiB (RSS), difference: 1.19 MiB | ||
| 5899 | 2026.08.18 00:00:49.479190 [ 432064 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 5900 | 2026.08.18 00:00:49.479189 [ 432098 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345576960, merge_with_ttl_allowedtrue) | ||
| 5901 | 2026.08.18 00:00:49.479212 [ 432098 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 5902 | 2026.08.18 00:00:52.895548 [ 432238 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Found 6 old parts to remove. Parts: [202608_1_1_0, 202608_2_2_0, 202608_3_3_0, 202608_4_4_0, 202608_5_5_0, 202608_6_6_0] | ||
| 5903 | 2026.08.18 00:00:52.895565 [ 432238 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from filesystem (serially): Parts: [202608_1_1_0, 202608_2_2_0, 202608_3_3_0, 202608_4_4_0, 202608_5_5_0, 202608_6_6_0] | ||
| 5904 | 2026.08.18 00:00:52.896251 [ 432238 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from memory: Parts: [202608_1_1_0, 202608_2_2_0, 202608_3_3_0, 202608_4_4_0, 202608_5_5_0, 202608_6_6_0] | ||
| 5905 | 2026.08.18 00:00:52.929522 [ 432234 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Found 6 old parts to remove. Parts: [202608_1_1_0, 202608_2_2_0, 202608_3_3_0, 202608_4_4_0, 202608_5_5_0, 202608_6_6_0] | ||
| 5906 | 2026.08.18 00:00:52.929533 [ 432234 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from filesystem (serially): Parts: [202608_1_1_0, 202608_2_2_0, 202608_3_3_0, 202608_4_4_0, 202608_5_5_0, 202608_6_6_0] | ||
| 5907 | 2026.08.18 00:00:52.930329 [ 432234 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from memory: Parts: [202608_1_1_0, 202608_2_2_0, 202608_3_3_0, 202608_4_4_0, 202608_5_5_0, 202608_6_6_0] | ||
| 5908 | 2026.08.18 00:00:52.996470 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2256 entries to flush up to offset 163755 | ||
| 5909 | 2026.08.18 00:00:52.997480 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5910 | 2026.08.18 00:00:52.997493 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5911 | 2026.08.18 00:00:52.997937 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_69_69_0 (state Temporary) | ||
| 5912 | 2026.08.18 00:00:52.998060 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_69_69_0 to 202608_70_70_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5913 | 2026.08.18 00:00:52.998131 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 163755 | ||
| 5914 | 2026.08.18 00:00:53.000238 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 562.09 MiB, peak 590.72 MiB, free memory in arenas 131.96 MiB, will set to 566.21 MiB (RSS), difference: 4.11 MiB | ||
| 5915 | 2026.08.18 00:00:53.843138 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 519 | ||
| 5916 | 2026.08.18 00:00:53.852151 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5917 | 2026.08.18 00:00:53.854552 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5918 | 2026.08.18 00:00:53.858018 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_69_69_0 (state Temporary) | ||
| 5919 | 2026.08.18 00:00:53.860609 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_69_69_0 to 202608_70_70_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5920 | 2026.08.18 00:00:53.861388 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 519 | ||
| 5921 | 2026.08.18 00:00:54.000162 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 566.92 MiB, peak 590.72 MiB, free memory in arenas 141.44 MiB, will set to 565.60 MiB (RSS), difference: -1.32 MiB | ||
| 5922 | 2026.08.18 00:00:54.109622 [ 432126 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346531328, merge_with_ttl_allowedtrue) | ||
| 5923 | 2026.08.18 00:00:54.109633 [ 432126 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 5924 | 2026.08.18 00:00:54.194363 [ 432139 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346531328, merge_with_ttl_allowedtrue) | ||
| 5925 | 2026.08.18 00:00:54.194372 [ 432139 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 5926 | 2026.08.18 00:01:00.329971 [ 432145 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5927 | 2026.08.18 00:01:00.329995 [ 432145 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5928 | 2026.08.18 00:01:00.498301 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 166331 | ||
| 5929 | 2026.08.18 00:01:00.499346 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5930 | 2026.08.18 00:01:00.499357 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5931 | 2026.08.18 00:01:00.499691 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_70_70_0 (state Temporary) | ||
| 5932 | 2026.08.18 00:01:00.499894 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_70_70_0 to 202608_71_71_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5933 | 2026.08.18 00:01:00.499974 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 166331 | ||
| 5934 | 2026.08.18 00:01:00.499977 [ 432152 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_66_13 to 202608_71_71_0 | ||
| 5935 | 2026.08.18 00:01:00.499993 [ 432152 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5936 | 2026.08.18 00:01:00.500002 [ 432152 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5937 | 2026.08.18 00:01:00.500109 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_66_13 to 202608_71_71_0 into Compact with storage Full | ||
| 5938 | 2026.08.18 00:01:00.500199 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5939 | 2026.08.18 00:01:00.500220 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 21 marks from part 202608_1_66_13, total 156453 rows starting from the beginning of the part | ||
| 5940 | 2026.08.18 00:01:00.500267 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_67_67_0, total 2576 rows starting from the beginning of the part | ||
| 5941 | 2026.08.18 00:01:00.500302 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_68_68_0, total 2254 rows starting from the beginning of the part | ||
| 5942 | 2026.08.18 00:01:00.500318 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_69_69_0, total 2576 rows starting from the beginning of the part | ||
| 5943 | 2026.08.18 00:01:00.500337 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_70_70_0, total 2256 rows starting from the beginning of the part | ||
| 5944 | 2026.08.18 00:01:00.500354 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_71_71_0, total 2576 rows starting from the beginning of the part | ||
| 5945 | 2026.08.18 00:01:00.511931 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 168691 rows, containing 4 columns (4 merged, 0 gathered) in 0.011871718 sec., 14209485.097270673 rows/sec., 204.25 MiB/sec. | ||
| 5946 | 2026.08.18 00:01:00.512180 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Trace> MergedBlockOutputStream: filled checksums 202608_1_71_14 (state Temporary) | ||
| 5947 | 2026.08.18 00:01:00.512331 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_71_14 to 202608_1_71_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5948 | 2026.08.18 00:01:00.512359 [ 432207 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_71_14} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_66_13, 202608_71_71_0] -> 202608_1_71_14 | ||
| 5949 | 2026.08.18 00:01:00.512412 [ 432207 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 5950 | 2026.08.18 00:01:01.361557 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 527 | ||
| 5951 | 2026.08.18 00:01:01.369960 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5952 | 2026.08.18 00:01:01.369976 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5953 | 2026.08.18 00:01:01.373529 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_70_70_0 (state Temporary) | ||
| 5954 | 2026.08.18 00:01:01.376034 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_70_70_0 to 202608_71_71_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5955 | 2026.08.18 00:01:01.376169 [ 432160 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_66_13 to 202608_71_71_0 | ||
| 5956 | 2026.08.18 00:01:01.376187 [ 432160 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5957 | 2026.08.18 00:01:01.376191 [ 432160 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5958 | 2026.08.18 00:01:01.376269 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_66_13 to 202608_71_71_0 into Compact with storage Full | ||
| 5959 | 2026.08.18 00:01:01.376833 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 527 | ||
| 5960 | 2026.08.18 00:01:01.379391 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 5961 | 2026.08.18 00:01:01.379786 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_66_13, total 497 rows starting from the beginning of the part | ||
| 5962 | 2026.08.18 00:01:01.381689 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_67_67_0, total 7 rows starting from the beginning of the part | ||
| 5963 | 2026.08.18 00:01:01.383564 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_68_68_0, total 8 rows starting from the beginning of the part | ||
| 5964 | 2026.08.18 00:01:01.385483 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_69_69_0, total 7 rows starting from the beginning of the part | ||
| 5965 | 2026.08.18 00:01:01.387320 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_70_70_0, total 8 rows starting from the beginning of the part | ||
| 5966 | 2026.08.18 00:01:01.389160 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_71_71_0, total 8 rows starting from the beginning of the part | ||
| 5967 | 2026.08.18 00:01:01.417921 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 535 rows, containing 679 columns (679 merged, 0 gathered) in 0.04168738 sec., 12833.620150750658 rows/sec., 7.91 MiB/sec. | ||
| 5968 | 2026.08.18 00:01:01.418859 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Trace> MergedBlockOutputStream: filled checksums 202608_1_71_14 (state Temporary) | ||
| 5969 | 2026.08.18 00:01:01.422335 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_71_14 to 202608_1_71_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5970 | 2026.08.18 00:01:01.422371 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_71_14} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_66_13, 202608_71_71_0] -> 202608_1_71_14 | ||
| 5971 | 2026.08.18 00:01:01.422803 [ 432216 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.47 MiB. | ||
| 5972 | 2026.08.18 00:01:02.000283 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 567.25 MiB, peak 590.72 MiB, free memory in arenas 162.14 MiB, will set to 583.18 MiB (RSS), difference: 15.93 MiB | ||
| 5973 | 2026.08.18 00:01:08.000057 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 168585 | ||
| 5974 | 2026.08.18 00:01:08.001038 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5975 | 2026.08.18 00:01:08.001051 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5976 | 2026.08.18 00:01:08.001495 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_71_71_0 (state Temporary) | ||
| 5977 | 2026.08.18 00:01:08.001625 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_71_71_0 to 202608_72_72_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5978 | 2026.08.18 00:01:08.001703 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 168585 | ||
| 5979 | 2026.08.18 00:01:08.876920 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 534 | ||
| 5980 | 2026.08.18 00:01:08.884876 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5981 | 2026.08.18 00:01:08.884892 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5982 | 2026.08.18 00:01:08.888307 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_71_71_0 (state Temporary) | ||
| 5983 | 2026.08.18 00:01:08.890779 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_71_71_0 to 202608_72_72_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5984 | 2026.08.18 00:01:08.891570 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 534 | ||
| 5985 | 2026.08.18 00:01:09.000260 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 583.91 MiB, peak 591.93 MiB, free memory in arenas 163.24 MiB, will set to 585.06 MiB (RSS), difference: 1.15 MiB | ||
| 5986 | 2026.08.18 00:01:15.330090 [ 432097 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 5987 | 2026.08.18 00:01:15.330121 [ 432097 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 5988 | 2026.08.18 00:01:15.501940 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 171161 | ||
| 5989 | 2026.08.18 00:01:15.503024 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5990 | 2026.08.18 00:01:15.503036 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5991 | 2026.08.18 00:01:15.503522 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_72_72_0 (state Temporary) | ||
| 5992 | 2026.08.18 00:01:15.503654 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_72_72_0 to 202608_73_73_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5993 | 2026.08.18 00:01:15.503736 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 171161 | ||
| 5994 | 2026.08.18 00:01:16.391655 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 542 | ||
| 5995 | 2026.08.18 00:01:16.399670 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 5996 | 2026.08.18 00:01:16.399686 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 5997 | 2026.08.18 00:01:16.403144 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_72_72_0 (state Temporary) | ||
| 5998 | 2026.08.18 00:01:16.405761 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_72_72_0 to 202608_73_73_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 5999 | 2026.08.18 00:01:16.406547 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 542 | ||
| 6000 | 2026.08.18 00:01:19.000238 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 585.22 MiB, peak 594.12 MiB, free memory in arenas 162.14 MiB, will set to 574.24 MiB (RSS), difference: -10.98 MiB | ||
| 6001 | 2026.08.18 00:01:23.003820 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 173737 | ||
| 6002 | 2026.08.18 00:01:23.004815 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6003 | 2026.08.18 00:01:23.004829 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6004 | 2026.08.18 00:01:23.005141 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_73_73_0 (state Temporary) | ||
| 6005 | 2026.08.18 00:01:23.005262 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_73_73_0 to 202608_74_74_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6006 | 2026.08.18 00:01:23.005337 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 173737 | ||
| 6007 | 2026.08.18 00:01:23.906649 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 549 | ||
| 6008 | 2026.08.18 00:01:23.918018 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6009 | 2026.08.18 00:01:23.918039 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6010 | 2026.08.18 00:01:23.922372 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_73_73_0 (state Temporary) | ||
| 6011 | 2026.08.18 00:01:23.925381 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_73_73_0 to 202608_74_74_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6012 | 2026.08.18 00:01:23.926229 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 549 | ||
| 6013 | 2026.08.18 00:01:30.330301 [ 432121 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6014 | 2026.08.18 00:01:30.330333 [ 432121 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6015 | 2026.08.18 00:01:30.505512 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 175991 | ||
| 6016 | 2026.08.18 00:01:30.506463 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6017 | 2026.08.18 00:01:30.506475 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6018 | 2026.08.18 00:01:30.506801 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_74_74_0 (state Temporary) | ||
| 6019 | 2026.08.18 00:01:30.506927 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_74_74_0 to 202608_75_75_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6020 | 2026.08.18 00:01:30.507008 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 175991 | ||
| 6021 | 2026.08.18 00:01:30.507073 [ 432236 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Found 6 old parts to remove. Parts: [202608_1_6_1, 202608_7_7_0, 202608_8_8_0, 202608_9_9_0, 202608_10_10_0, 202608_11_11_0] | ||
| 6022 | 2026.08.18 00:01:30.507084 [ 432236 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from filesystem (serially): Parts: [202608_1_6_1, 202608_7_7_0, 202608_8_8_0, 202608_9_9_0, 202608_10_10_0, 202608_11_11_0] | ||
| 6023 | 2026.08.18 00:01:30.507809 [ 432236 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from memory: Parts: [202608_1_6_1, 202608_7_7_0, 202608_8_8_0, 202608_9_9_0, 202608_10_10_0, 202608_11_11_0] | ||
| 6024 | 2026.08.18 00:01:30.818431 [ 432238 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Found 6 old parts to remove. Parts: [202608_1_6_1, 202608_7_7_0, 202608_8_8_0, 202608_9_9_0, 202608_10_10_0, 202608_11_11_0] | ||
| 6025 | 2026.08.18 00:01:30.818442 [ 432238 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from filesystem (serially): Parts: [202608_1_6_1, 202608_7_7_0, 202608_8_8_0, 202608_9_9_0, 202608_10_10_0, 202608_11_11_0] | ||
| 6026 | 2026.08.18 00:01:30.819178 [ 432238 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from memory: Parts: [202608_1_6_1, 202608_7_7_0, 202608_8_8_0, 202608_9_9_0, 202608_10_10_0, 202608_11_11_0] | ||
| 6027 | 2026.08.18 00:01:31.426329 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 557 | ||
| 6028 | 2026.08.18 00:01:31.433642 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6029 | 2026.08.18 00:01:31.433657 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6030 | 2026.08.18 00:01:31.436973 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_74_74_0 (state Temporary) | ||
| 6031 | 2026.08.18 00:01:31.439475 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_74_74_0 to 202608_75_75_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6032 | 2026.08.18 00:01:31.440252 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 557 | ||
| 6033 | 2026.08.18 00:01:32.059633 [ 432084 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346230272, merge_with_ttl_allowedtrue) | ||
| 6034 | 2026.08.18 00:01:32.059646 [ 432084 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 6035 | 2026.08.18 00:01:38.007094 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 178567 | ||
| 6036 | 2026.08.18 00:01:38.008273 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6037 | 2026.08.18 00:01:38.008287 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6038 | 2026.08.18 00:01:38.008753 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_75_75_0 (state Temporary) | ||
| 6039 | 2026.08.18 00:01:38.008883 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_75_75_0 to 202608_76_76_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6040 | 2026.08.18 00:01:38.008966 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 178567 | ||
| 6041 | 2026.08.18 00:01:38.008972 [ 432154 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_71_14 to 202608_76_76_0 | ||
| 6042 | 2026.08.18 00:01:38.008987 [ 432154 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6043 | 2026.08.18 00:01:38.008992 [ 432154 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6044 | 2026.08.18 00:01:38.009113 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_71_14 to 202608_76_76_0 into Compact with storage Full | ||
| 6045 | 2026.08.18 00:01:38.009191 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6046 | 2026.08.18 00:01:38.009211 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 22 marks from part 202608_1_71_14, total 168691 rows starting from the beginning of the part | ||
| 6047 | 2026.08.18 00:01:38.009253 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_72_72_0, total 2254 rows starting from the beginning of the part | ||
| 6048 | 2026.08.18 00:01:38.009274 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_73_73_0, total 2576 rows starting from the beginning of the part | ||
| 6049 | 2026.08.18 00:01:38.009317 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_74_74_0, total 2576 rows starting from the beginning of the part | ||
| 6050 | 2026.08.18 00:01:38.009345 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_75_75_0, total 2254 rows starting from the beginning of the part | ||
| 6051 | 2026.08.18 00:01:38.009362 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_76_76_0, total 2576 rows starting from the beginning of the part | ||
| 6052 | 2026.08.18 00:01:38.022011 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 180927 rows, containing 4 columns (4 merged, 0 gathered) in 0.012944229 sec., 13977425.770202305 rows/sec., 200.85 MiB/sec. | ||
| 6053 | 2026.08.18 00:01:38.022132 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Trace> MergedBlockOutputStream: filled checksums 202608_1_76_15 (state Temporary) | ||
| 6054 | 2026.08.18 00:01:38.022266 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_76_15 to 202608_1_76_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6055 | 2026.08.18 00:01:38.022291 [ 432226 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_76_15} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_71_14, 202608_76_76_0] -> 202608_1_76_15 | ||
| 6056 | 2026.08.18 00:01:38.022337 [ 432226 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.79 MiB. | ||
| 6057 | 2026.08.18 00:01:38.940426 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 564 | ||
| 6058 | 2026.08.18 00:01:38.948401 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6059 | 2026.08.18 00:01:38.948415 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6060 | 2026.08.18 00:01:38.951715 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_75_75_0 (state Temporary) | ||
| 6061 | 2026.08.18 00:01:38.954201 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_75_75_0 to 202608_76_76_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6062 | 2026.08.18 00:01:38.954343 [ 432160 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_71_14 to 202608_76_76_0 | ||
| 6063 | 2026.08.18 00:01:38.954360 [ 432160 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6064 | 2026.08.18 00:01:38.954365 [ 432160 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6065 | 2026.08.18 00:01:38.954471 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_71_14 to 202608_76_76_0 into Compact with storage Full | ||
| 6066 | 2026.08.18 00:01:38.955074 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 564 | ||
| 6067 | 2026.08.18 00:01:38.958213 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6068 | 2026.08.18 00:01:38.958593 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_71_14, total 535 rows starting from the beginning of the part | ||
| 6069 | 2026.08.18 00:01:38.960498 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_72_72_0, total 7 rows starting from the beginning of the part | ||
| 6070 | 2026.08.18 00:01:38.962373 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_73_73_0, total 8 rows starting from the beginning of the part | ||
| 6071 | 2026.08.18 00:01:38.964306 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_74_74_0, total 7 rows starting from the beginning of the part | ||
| 6072 | 2026.08.18 00:01:38.966153 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_75_75_0, total 8 rows starting from the beginning of the part | ||
| 6073 | 2026.08.18 00:01:38.967994 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_76_76_0, total 7 rows starting from the beginning of the part | ||
| 6074 | 2026.08.18 00:01:38.997088 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 572 rows, containing 679 columns (679 merged, 0 gathered) in 0.042673241 sec., 13404.184603648924 rows/sec., 8.07 MiB/sec. | ||
| 6075 | 2026.08.18 00:01:38.998073 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Trace> MergedBlockOutputStream: filled checksums 202608_1_76_15 (state Temporary) | ||
| 6076 | 2026.08.18 00:01:39.000372 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 578.62 MiB, peak 594.12 MiB, free memory in arenas 146.42 MiB, will set to 583.71 MiB (RSS), difference: 5.09 MiB | ||
| 6077 | 2026.08.18 00:01:39.001460 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_76_15 to 202608_1_76_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6078 | 2026.08.18 00:01:39.001507 [ 432202 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_76_15} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_71_14, 202608_76_76_0] -> 202608_1_76_15 | ||
| 6079 | 2026.08.18 00:01:39.001988 [ 432202 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.63 MiB. | ||
| 6080 | 2026.08.18 00:01:40.000362 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 578.31 MiB, peak 594.12 MiB, free memory in arenas 154.59 MiB, will set to 583.71 MiB (RSS), difference: 5.40 MiB | ||
| 6081 | 2026.08.18 00:01:45.330528 [ 432091 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6082 | 2026.08.18 00:01:45.330561 [ 432091 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6083 | 2026.08.18 00:01:45.509132 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 180821 | ||
| 6084 | 2026.08.18 00:01:45.510980 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6085 | 2026.08.18 00:01:45.510993 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6086 | 2026.08.18 00:01:45.511342 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_76_76_0 (state Temporary) | ||
| 6087 | 2026.08.18 00:01:45.511472 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_76_76_0 to 202608_77_77_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6088 | 2026.08.18 00:01:45.511552 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 180821 | ||
| 6089 | 2026.08.18 00:01:46.455172 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 572 | ||
| 6090 | 2026.08.18 00:01:46.463719 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6091 | 2026.08.18 00:01:46.463737 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6092 | 2026.08.18 00:01:46.467254 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_76_76_0 (state Temporary) | ||
| 6093 | 2026.08.18 00:01:46.469900 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_76_76_0 to 202608_77_77_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6094 | 2026.08.18 00:01:46.470689 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 572 | ||
| 6095 | 2026.08.18 00:01:47.000178 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 584.27 MiB, peak 594.12 MiB, free memory in arenas 144.06 MiB, will set to 581.95 MiB (RSS), difference: -2.32 MiB | ||
| 6096 | 2026.08.18 00:01:49.479274 [ 432106 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 6097 | 2026.08.18 00:01:49.479286 [ 432096 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345667072, merge_with_ttl_allowedtrue) | ||
| 6098 | 2026.08.18 00:01:49.479295 [ 432096 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 6099 | 2026.08.18 00:01:53.011636 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 183397 | ||
| 6100 | 2026.08.18 00:01:53.013573 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6101 | 2026.08.18 00:01:53.013586 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6102 | 2026.08.18 00:01:53.013967 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_77_77_0 (state Temporary) | ||
| 6103 | 2026.08.18 00:01:53.014095 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_77_77_0 to 202608_78_78_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6104 | 2026.08.18 00:01:53.014178 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 183397 | ||
| 6105 | 2026.08.18 00:01:53.970772 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 579 | ||
| 6106 | 2026.08.18 00:01:53.979187 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6107 | 2026.08.18 00:01:53.979202 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6108 | 2026.08.18 00:01:53.982648 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_77_77_0 (state Temporary) | ||
| 6109 | 2026.08.18 00:01:53.985259 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_77_77_0 to 202608_78_78_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6110 | 2026.08.18 00:01:53.986017 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 579 | ||
| 6111 | 2026.08.18 00:01:54.000272 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 582.01 MiB, peak 594.12 MiB, free memory in arenas 143.28 MiB, will set to 569.15 MiB (RSS), difference: -12.86 MiB | ||
| 6112 | 2026.08.18 00:01:54.109843 [ 432061 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345337344, merge_with_ttl_allowedtrue) | ||
| 6113 | 2026.08.18 00:01:54.109851 [ 432061 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 6114 | 2026.08.18 00:01:54.194570 [ 432099 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345337344, merge_with_ttl_allowedtrue) | ||
| 6115 | 2026.08.18 00:01:54.194580 [ 432099 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 6116 | 2026.08.18 00:02:00.330684 [ 432098 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6117 | 2026.08.18 00:02:00.330724 [ 432098 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6118 | 2026.08.18 00:02:00.514677 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 185651 | ||
| 6119 | 2026.08.18 00:02:00.516377 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6120 | 2026.08.18 00:02:00.516399 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6121 | 2026.08.18 00:02:00.517139 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_78_78_0 (state Temporary) | ||
| 6122 | 2026.08.18 00:02:00.517348 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_78_78_0 to 202608_79_79_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6123 | 2026.08.18 00:02:00.517468 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 185651 | ||
| 6124 | 2026.08.18 00:02:01.000282 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 569.15 MiB, peak 594.12 MiB, free memory in arenas 143.29 MiB, will set to 564.85 MiB (RSS), difference: -4.30 MiB | ||
| 6125 | 2026.08.18 00:02:01.486118 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 587 | ||
| 6126 | 2026.08.18 00:02:01.498946 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6127 | 2026.08.18 00:02:01.498966 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6128 | 2026.08.18 00:02:01.503298 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_78_78_0 (state Temporary) | ||
| 6129 | 2026.08.18 00:02:01.506276 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_78_78_0 to 202608_79_79_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6130 | 2026.08.18 00:02:01.507091 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 587 | ||
| 6131 | 2026.08.18 00:02:07.272596 [ 432238 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Found 6 old parts to remove. Parts: [202608_1_11_2, 202608_12_12_0, 202608_13_13_0, 202608_14_14_0, 202608_15_15_0, 202608_16_16_0] | ||
| 6132 | 2026.08.18 00:02:07.272612 [ 432238 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from filesystem (serially): Parts: [202608_1_11_2, 202608_12_12_0, 202608_13_13_0, 202608_14_14_0, 202608_15_15_0, 202608_16_16_0] | ||
| 6133 | 2026.08.18 00:02:07.273255 [ 432238 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from memory: Parts: [202608_1_11_2, 202608_12_12_0, 202608_13_13_0, 202608_14_14_0, 202608_15_15_0, 202608_16_16_0] | ||
| 6134 | 2026.08.18 00:02:08.017650 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 188227 | ||
| 6135 | 2026.08.18 00:02:08.018731 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6136 | 2026.08.18 00:02:08.018743 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6137 | 2026.08.18 00:02:08.019238 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_79_79_0 (state Temporary) | ||
| 6138 | 2026.08.18 00:02:08.019372 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_79_79_0 to 202608_80_80_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6139 | 2026.08.18 00:02:08.019453 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 188227 | ||
| 6140 | 2026.08.18 00:02:08.182621 [ 432234 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Found 6 old parts to remove. Parts: [202608_1_11_2, 202608_12_12_0, 202608_13_13_0, 202608_14_14_0, 202608_15_15_0, 202608_16_16_0] | ||
| 6141 | 2026.08.18 00:02:08.182636 [ 432234 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from filesystem (serially): Parts: [202608_1_11_2, 202608_12_12_0, 202608_13_13_0, 202608_14_14_0, 202608_15_15_0, 202608_16_16_0] | ||
| 6142 | 2026.08.18 00:02:08.183411 [ 432234 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from memory: Parts: [202608_1_11_2, 202608_12_12_0, 202608_13_13_0, 202608_14_14_0, 202608_15_15_0, 202608_16_16_0] | ||
| 6143 | 2026.08.18 00:02:09.000300 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 561.52 MiB, peak 594.12 MiB, free memory in arenas 146.79 MiB, will set to 565.54 MiB (RSS), difference: 4.01 MiB | ||
| 6144 | 2026.08.18 00:02:09.007173 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 594 | ||
| 6145 | 2026.08.18 00:02:09.014198 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6146 | 2026.08.18 00:02:09.014218 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6147 | 2026.08.18 00:02:09.017751 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_79_79_0 (state Temporary) | ||
| 6148 | 2026.08.18 00:02:09.020317 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_79_79_0 to 202608_80_80_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6149 | 2026.08.18 00:02:09.021094 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 594 | ||
| 6150 | 2026.08.18 00:02:15.330907 [ 432158 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6151 | 2026.08.18 00:02:15.330937 [ 432158 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6152 | 2026.08.18 00:02:15.519527 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 190481 | ||
| 6153 | 2026.08.18 00:02:15.520479 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6154 | 2026.08.18 00:02:15.520491 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6155 | 2026.08.18 00:02:15.521050 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_80_80_0 (state Temporary) | ||
| 6156 | 2026.08.18 00:02:15.521179 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_80_80_0 to 202608_81_81_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6157 | 2026.08.18 00:02:15.521262 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 190481 | ||
| 6158 | 2026.08.18 00:02:15.521278 [ 432140 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_76_15 to 202608_81_81_0 | ||
| 6159 | 2026.08.18 00:02:15.521297 [ 432140 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6160 | 2026.08.18 00:02:15.521303 [ 432140 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6161 | 2026.08.18 00:02:15.521391 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_76_15 to 202608_81_81_0 into Compact with storage Full | ||
| 6162 | 2026.08.18 00:02:15.521461 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6163 | 2026.08.18 00:02:15.521481 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 24 marks from part 202608_1_76_15, total 180927 rows starting from the beginning of the part | ||
| 6164 | 2026.08.18 00:02:15.521526 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_77_77_0, total 2254 rows starting from the beginning of the part | ||
| 6165 | 2026.08.18 00:02:15.521553 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_78_78_0, total 2576 rows starting from the beginning of the part | ||
| 6166 | 2026.08.18 00:02:15.521569 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_79_79_0, total 2254 rows starting from the beginning of the part | ||
| 6167 | 2026.08.18 00:02:15.521585 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_80_80_0, total 2576 rows starting from the beginning of the part | ||
| 6168 | 2026.08.18 00:02:15.521603 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_81_81_0, total 2254 rows starting from the beginning of the part | ||
| 6169 | 2026.08.18 00:02:15.534537 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 192841 rows, containing 4 columns (4 merged, 0 gathered) in 0.013184989 sec., 14625799.081061047 rows/sec., 210.09 MiB/sec. | ||
| 6170 | 2026.08.18 00:02:15.534701 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Trace> MergedBlockOutputStream: filled checksums 202608_1_81_16 (state Temporary) | ||
| 6171 | 2026.08.18 00:02:15.534834 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_81_16 to 202608_1_81_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6172 | 2026.08.18 00:02:15.534858 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_81_16} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_76_15, 202608_81_81_0] -> 202608_1_81_16 | ||
| 6173 | 2026.08.18 00:02:15.534902 [ 432210 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 6174 | 2026.08.18 00:02:16.000257 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 566.15 MiB, peak 594.12 MiB, free memory in arenas 136.96 MiB, will set to 560.45 MiB (RSS), difference: -5.70 MiB | ||
| 6175 | 2026.08.18 00:02:16.000756 [ 432270 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 7.065e-05 sec. | ||
| 6176 | 2026.08.18 00:02:16.521255 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 602 | ||
| 6177 | 2026.08.18 00:02:16.529005 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6178 | 2026.08.18 00:02:16.529019 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6179 | 2026.08.18 00:02:16.532328 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_80_80_0 (state Temporary) | ||
| 6180 | 2026.08.18 00:02:16.534803 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_80_80_0 to 202608_81_81_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6181 | 2026.08.18 00:02:16.534953 [ 432157 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_76_15 to 202608_81_81_0 | ||
| 6182 | 2026.08.18 00:02:16.534966 [ 432157 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6183 | 2026.08.18 00:02:16.534971 [ 432157 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6184 | 2026.08.18 00:02:16.535054 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_76_15 to 202608_81_81_0 into Compact with storage Full | ||
| 6185 | 2026.08.18 00:02:16.535669 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 602 | ||
| 6186 | 2026.08.18 00:02:16.539066 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6187 | 2026.08.18 00:02:16.539395 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_76_15, total 572 rows starting from the beginning of the part | ||
| 6188 | 2026.08.18 00:02:16.541197 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_77_77_0, total 8 rows starting from the beginning of the part | ||
| 6189 | 2026.08.18 00:02:16.543025 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_78_78_0, total 7 rows starting from the beginning of the part | ||
| 6190 | 2026.08.18 00:02:16.545021 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_79_79_0, total 8 rows starting from the beginning of the part | ||
| 6191 | 2026.08.18 00:02:16.546834 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_80_80_0, total 7 rows starting from the beginning of the part | ||
| 6192 | 2026.08.18 00:02:16.548619 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_81_81_0, total 8 rows starting from the beginning of the part | ||
| 6193 | 2026.08.18 00:02:16.576910 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 610 rows, containing 679 columns (679 merged, 0 gathered) in 0.04189316 sec., 14560.849551573574 rows/sec., 8.69 MiB/sec. | ||
| 6194 | 2026.08.18 00:02:16.577842 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Trace> MergedBlockOutputStream: filled checksums 202608_1_81_16 (state Temporary) | ||
| 6195 | 2026.08.18 00:02:16.581154 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_81_16 to 202608_1_81_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6196 | 2026.08.18 00:02:16.581194 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_81_16} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_76_15, 202608_81_81_0] -> 202608_1_81_16 | ||
| 6197 | 2026.08.18 00:02:16.581880 [ 432225 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.63 MiB. | ||
| 6198 | 2026.08.18 00:02:17.000409 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 561.91 MiB, peak 594.12 MiB, free memory in arenas 139.43 MiB, will set to 567.13 MiB (RSS), difference: 5.21 MiB | ||
| 6199 | 2026.08.18 00:02:17.000570 [ 432270 ] {} <Trace> AsynchronousMetrics: Scanning /sys/block | ||
| 6200 | 2026.08.18 00:02:23.021518 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 193057 | ||
| 6201 | 2026.08.18 00:02:23.022636 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6202 | 2026.08.18 00:02:23.022649 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6203 | 2026.08.18 00:02:23.023115 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_81_81_0 (state Temporary) | ||
| 6204 | 2026.08.18 00:02:23.023254 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_81_81_0 to 202608_82_82_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6205 | 2026.08.18 00:02:23.023336 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 193057 | ||
| 6206 | 2026.08.18 00:02:24.035854 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 609 | ||
| 6207 | 2026.08.18 00:02:24.044288 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6208 | 2026.08.18 00:02:24.044303 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6209 | 2026.08.18 00:02:24.047754 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_81_81_0 (state Temporary) | ||
| 6210 | 2026.08.18 00:02:24.050397 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_81_81_0 to 202608_82_82_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6211 | 2026.08.18 00:02:24.051163 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 609 | ||
| 6212 | 2026.08.18 00:02:25.000295 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 567.72 MiB, peak 594.12 MiB, free memory in arenas 139.67 MiB, will set to 566.16 MiB (RSS), difference: -1.56 MiB | ||
| 6213 | 2026.08.18 00:02:30.331121 [ 432019 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6214 | 2026.08.18 00:02:30.331146 [ 432019 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6215 | 2026.08.18 00:02:30.523509 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 195311 | ||
| 6216 | 2026.08.18 00:02:30.524447 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6217 | 2026.08.18 00:02:30.524459 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6218 | 2026.08.18 00:02:30.524816 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_82_82_0 (state Temporary) | ||
| 6219 | 2026.08.18 00:02:30.524941 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_82_82_0 to 202608_83_83_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6220 | 2026.08.18 00:02:30.525024 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 195311 | ||
| 6221 | 2026.08.18 00:02:31.551329 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 617 | ||
| 6222 | 2026.08.18 00:02:31.558900 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6223 | 2026.08.18 00:02:31.558915 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6224 | 2026.08.18 00:02:31.562321 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_82_82_0 (state Temporary) | ||
| 6225 | 2026.08.18 00:02:31.564809 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_82_82_0 to 202608_83_83_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6226 | 2026.08.18 00:02:31.565569 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 617 | ||
| 6227 | 2026.08.18 00:02:32.059844 [ 432062 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345032192, merge_with_ttl_allowedtrue) | ||
| 6228 | 2026.08.18 00:02:32.059865 [ 432062 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 6229 | 2026.08.18 00:02:38.025263 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 197887 | ||
| 6230 | 2026.08.18 00:02:38.026259 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6231 | 2026.08.18 00:02:38.026271 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6232 | 2026.08.18 00:02:38.026722 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_83_83_0 (state Temporary) | ||
| 6233 | 2026.08.18 00:02:38.026847 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_83_83_0 to 202608_84_84_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6234 | 2026.08.18 00:02:38.026925 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 197887 | ||
| 6235 | 2026.08.18 00:02:39.065732 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 624 | ||
| 6236 | 2026.08.18 00:02:39.073075 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6237 | 2026.08.18 00:02:39.073090 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6238 | 2026.08.18 00:02:39.076587 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_83_83_0 (state Temporary) | ||
| 6239 | 2026.08.18 00:02:39.079146 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_83_83_0 to 202608_84_84_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6240 | 2026.08.18 00:02:39.079915 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 624 | ||
| 6241 | 2026.08.18 00:02:45.151060 [ 432237 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Found 6 old parts to remove. Parts: [202608_1_16_3, 202608_17_17_0, 202608_18_18_0, 202608_19_19_0, 202608_20_20_0, 202608_21_21_0] | ||
| 6242 | 2026.08.18 00:02:45.151072 [ 432237 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from filesystem (serially): Parts: [202608_1_16_3, 202608_17_17_0, 202608_18_18_0, 202608_19_19_0, 202608_20_20_0, 202608_21_21_0] | ||
| 6243 | 2026.08.18 00:02:45.151740 [ 432237 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from memory: Parts: [202608_1_16_3, 202608_17_17_0, 202608_18_18_0, 202608_19_19_0, 202608_20_20_0, 202608_21_21_0] | ||
| 6244 | 2026.08.18 00:02:45.296371 [ 432236 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Found 6 old parts to remove. Parts: [202608_1_16_3, 202608_17_17_0, 202608_18_18_0, 202608_19_19_0, 202608_20_20_0, 202608_21_21_0] | ||
| 6245 | 2026.08.18 00:02:45.296379 [ 432236 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from filesystem (serially): Parts: [202608_1_16_3, 202608_17_17_0, 202608_18_18_0, 202608_19_19_0, 202608_20_20_0, 202608_21_21_0] | ||
| 6246 | 2026.08.18 00:02:45.297078 [ 432236 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from memory: Parts: [202608_1_16_3, 202608_17_17_0, 202608_18_18_0, 202608_19_19_0, 202608_20_20_0, 202608_21_21_0] | ||
| 6247 | 2026.08.18 00:02:45.331327 [ 432040 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6248 | 2026.08.18 00:02:45.331358 [ 432040 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6249 | 2026.08.18 00:02:45.527091 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 200141 | ||
| 6250 | 2026.08.18 00:02:45.528074 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6251 | 2026.08.18 00:02:45.528087 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6252 | 2026.08.18 00:02:45.528427 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_84_84_0 (state Temporary) | ||
| 6253 | 2026.08.18 00:02:45.528555 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_84_84_0 to 202608_85_85_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6254 | 2026.08.18 00:02:45.528630 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 200141 | ||
| 6255 | 2026.08.18 00:02:46.000296 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 563.62 MiB, peak 594.12 MiB, free memory in arenas 104.14 MiB, will set to 558.63 MiB (RSS), difference: -4.98 MiB | ||
| 6256 | 2026.08.18 00:02:46.580087 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 632 | ||
| 6257 | 2026.08.18 00:02:46.587905 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6258 | 2026.08.18 00:02:46.587919 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6259 | 2026.08.18 00:02:46.591577 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_84_84_0 (state Temporary) | ||
| 6260 | 2026.08.18 00:02:46.594454 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_84_84_0 to 202608_85_85_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6261 | 2026.08.18 00:02:46.595259 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 632 | ||
| 6262 | 2026.08.18 00:02:49.479346 [ 432127 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 6263 | 2026.08.18 00:02:49.479360 [ 432155 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345624064, merge_with_ttl_allowedtrue) | ||
| 6264 | 2026.08.18 00:02:49.479378 [ 432155 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 6265 | 2026.08.18 00:02:50.306751 [ 432120 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Found 5 old parts to remove. Parts: [all_0_0_0, all_1_1_0, all_2_2_0, all_3_3_0, all_4_4_0] | ||
| 6266 | 2026.08.18 00:02:50.306761 [ 432120 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Removing 5 old parts from ZooKeeper | ||
| 6267 | 2026.08.18 00:02:50.310857 [ 432120 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Removed 5 old parts from ZooKeeper. Removing them from filesystem. | ||
| 6268 | 2026.08.18 00:02:50.310865 [ 432120 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Removing 5 parts from filesystem (serially): Parts: [all_0_0_0, all_1_1_0, all_2_2_0, all_3_3_0, all_4_4_0] | ||
| 6269 | 2026.08.18 00:02:50.311426 [ 432120 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Removing 5 parts from memory: Parts: [all_0_0_0, all_1_1_0, all_2_2_0, all_3_3_0, all_4_4_0] | ||
| 6270 | 2026.08.18 00:02:50.311431 [ 432120 ] {} <Debug> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Removed 5 old parts | ||
| 6271 | 2026.08.18 00:02:50.311775 [ 432120 ] {} <Trace> oximeter.fields_i64_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 1.2820513, interval: 300801ms, ratio: 0.008547009, points per minute: 0.25572747) | ||
| 6272 | 2026.08.18 00:02:53.028803 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 202717 | ||
| 6273 | 2026.08.18 00:02:53.029884 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6274 | 2026.08.18 00:02:53.029895 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6275 | 2026.08.18 00:02:53.030523 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_85_85_0 (state Temporary) | ||
| 6276 | 2026.08.18 00:02:53.030659 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_85_85_0 to 202608_86_86_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6277 | 2026.08.18 00:02:53.030746 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 202717 | ||
| 6278 | 2026.08.18 00:02:53.030773 [ 432048 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_81_16 to 202608_86_86_0 | ||
| 6279 | 2026.08.18 00:02:53.030806 [ 432048 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6280 | 2026.08.18 00:02:53.030812 [ 432048 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6281 | 2026.08.18 00:02:53.030978 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_81_16 to 202608_86_86_0 into Compact with storage Full | ||
| 6282 | 2026.08.18 00:02:53.031056 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6283 | 2026.08.18 00:02:53.031077 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 25 marks from part 202608_1_81_16, total 192841 rows starting from the beginning of the part | ||
| 6284 | 2026.08.18 00:02:53.031124 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_82_82_0, total 2576 rows starting from the beginning of the part | ||
| 6285 | 2026.08.18 00:02:53.031153 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_83_83_0, total 2254 rows starting from the beginning of the part | ||
| 6286 | 2026.08.18 00:02:53.031171 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_84_84_0, total 2576 rows starting from the beginning of the part | ||
| 6287 | 2026.08.18 00:02:53.031194 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_85_85_0, total 2254 rows starting from the beginning of the part | ||
| 6288 | 2026.08.18 00:02:53.031210 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_86_86_0, total 2576 rows starting from the beginning of the part | ||
| 6289 | 2026.08.18 00:02:53.045198 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 205077 rows, containing 4 columns (4 merged, 0 gathered) in 0.01429267 sec., 14348403.76220818 rows/sec., 206.07 MiB/sec. | ||
| 6290 | 2026.08.18 00:02:53.045317 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Trace> MergedBlockOutputStream: filled checksums 202608_1_86_17 (state Temporary) | ||
| 6291 | 2026.08.18 00:02:53.045490 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_86_17 to 202608_1_86_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6292 | 2026.08.18 00:02:53.045513 [ 432203 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_86_17} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_81_16, 202608_86_86_0] -> 202608_1_86_17 | ||
| 6293 | 2026.08.18 00:02:53.045557 [ 432203 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.78 MiB. | ||
| 6294 | 2026.08.18 00:02:54.000269 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 559.59 MiB, peak 594.12 MiB, free memory in arenas 124.02 MiB, will set to 548.53 MiB (RSS), difference: -11.06 MiB | ||
| 6295 | 2026.08.18 00:02:54.095425 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 639 | ||
| 6296 | 2026.08.18 00:02:54.102324 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6297 | 2026.08.18 00:02:54.102338 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6298 | 2026.08.18 00:02:54.105618 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_85_85_0 (state Temporary) | ||
| 6299 | 2026.08.18 00:02:54.108130 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_85_85_0 to 202608_86_86_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6300 | 2026.08.18 00:02:54.108283 [ 432021 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_81_16 to 202608_86_86_0 | ||
| 6301 | 2026.08.18 00:02:54.108297 [ 432021 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6302 | 2026.08.18 00:02:54.108301 [ 432021 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6303 | 2026.08.18 00:02:54.108372 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_81_16 to 202608_86_86_0 into Compact with storage Full | ||
| 6304 | 2026.08.18 00:02:54.108951 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 639 | ||
| 6305 | 2026.08.18 00:02:54.109960 [ 432082 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345286144, merge_with_ttl_allowedtrue) | ||
| 6306 | 2026.08.18 00:02:54.109968 [ 432082 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 6307 | 2026.08.18 00:02:54.111703 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6308 | 2026.08.18 00:02:54.112062 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_81_16, total 610 rows starting from the beginning of the part | ||
| 6309 | 2026.08.18 00:02:54.113983 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_82_82_0, total 7 rows starting from the beginning of the part | ||
| 6310 | 2026.08.18 00:02:54.115952 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_83_83_0, total 8 rows starting from the beginning of the part | ||
| 6311 | 2026.08.18 00:02:54.117843 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_84_84_0, total 7 rows starting from the beginning of the part | ||
| 6312 | 2026.08.18 00:02:54.119679 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_85_85_0, total 8 rows starting from the beginning of the part | ||
| 6313 | 2026.08.18 00:02:54.121509 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_86_86_0, total 7 rows starting from the beginning of the part | ||
| 6314 | 2026.08.18 00:02:54.150443 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 647 rows, containing 679 columns (679 merged, 0 gathered) in 0.04209886 sec., 15368.587177895077 rows/sec., 9.08 MiB/sec. | ||
| 6315 | 2026.08.18 00:02:54.151467 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Trace> MergedBlockOutputStream: filled checksums 202608_1_86_17 (state Temporary) | ||
| 6316 | 2026.08.18 00:02:54.154852 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_86_17 to 202608_1_86_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6317 | 2026.08.18 00:02:54.154888 [ 432227 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_86_17} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_81_16, 202608_86_86_0] -> 202608_1_86_17 | ||
| 6318 | 2026.08.18 00:02:54.155346 [ 432227 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.63 MiB. | ||
| 6319 | 2026.08.18 00:02:54.194770 [ 432091 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345712128, merge_with_ttl_allowedtrue) | ||
| 6320 | 2026.08.18 00:02:54.194784 [ 432091 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 6321 | 2026.08.18 00:02:55.000284 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 549.99 MiB, peak 594.12 MiB, free memory in arenas 146.14 MiB, will set to 567.47 MiB (RSS), difference: 17.48 MiB | ||
| 6322 | 2026.08.18 00:03:00.331461 [ 432015 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6323 | 2026.08.18 00:03:00.331494 [ 432015 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6324 | 2026.08.18 00:03:00.530840 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 204971 | ||
| 6325 | 2026.08.18 00:03:00.532629 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6326 | 2026.08.18 00:03:00.532653 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6327 | 2026.08.18 00:03:00.533394 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_86_86_0 (state Temporary) | ||
| 6328 | 2026.08.18 00:03:00.533597 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_86_86_0 to 202608_87_87_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6329 | 2026.08.18 00:03:00.533720 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 204971 | ||
| 6330 | 2026.08.18 00:03:00.953082 [ 432104 ] {} <Trace> oximeter.version (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 309161ms, ratio: 0, points per minute: 0) | ||
| 6331 | 2026.08.18 00:03:01.609124 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 647 | ||
| 6332 | 2026.08.18 00:03:01.617037 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6333 | 2026.08.18 00:03:01.617052 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6334 | 2026.08.18 00:03:01.620427 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_86_86_0 (state Temporary) | ||
| 6335 | 2026.08.18 00:03:01.622938 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_86_86_0 to 202608_87_87_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6336 | 2026.08.18 00:03:01.623750 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 647 | ||
| 6337 | 2026.08.18 00:03:02.621850 [ 432099 ] {} <Trace> oximeter.timeseries_schema_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 309823ms, ratio: 0, points per minute: 0) | ||
| 6338 | 2026.08.18 00:03:05.100833 [ 432017 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Found 5 old parts to remove. Parts: [all_0_0_0, all_1_1_0, all_2_2_0, all_3_3_0, all_4_4_0] | ||
| 6339 | 2026.08.18 00:03:05.100843 [ 432017 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removing 5 old parts from ZooKeeper | ||
| 6340 | 2026.08.18 00:03:05.117648 [ 432017 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removed 5 old parts from ZooKeeper. Removing them from filesystem. | ||
| 6341 | 2026.08.18 00:03:05.117653 [ 432017 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removing 5 parts from filesystem (serially): Parts: [all_0_0_0, all_1_1_0, all_2_2_0, all_3_3_0, all_4_4_0] | ||
| 6342 | 2026.08.18 00:03:05.118151 [ 432017 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removing 5 parts from memory: Parts: [all_0_0_0, all_1_1_0, all_2_2_0, all_3_3_0, all_4_4_0] | ||
| 6343 | 2026.08.18 00:03:05.118155 [ 432017 ] {} <Debug> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Removed 5 old parts | ||
| 6344 | 2026.08.18 00:03:05.118432 [ 432017 ] {} <Trace> oximeter.fields_uuid_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 1.2820513, interval: 305969ms, ratio: 0.008547009, points per minute: 0.2514081) | ||
| 6345 | 2026.08.18 00:03:07.832252 [ 432033 ] {} <Trace> oximeter.measurements_cumulativef64_local (ReplicatedMergeTreeCleanupThread): Scheduling next cleanup after 300000ms (points: 0, interval: 309566ms, ratio: 0, points per minute: 0) | ||
| 6346 | 2026.08.18 00:03:08.033895 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 207547 | ||
| 6347 | 2026.08.18 00:03:08.035107 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6348 | 2026.08.18 00:03:08.035119 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6349 | 2026.08.18 00:03:08.035596 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_87_87_0 (state Temporary) | ||
| 6350 | 2026.08.18 00:03:08.035728 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_87_87_0 to 202608_88_88_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6351 | 2026.08.18 00:03:08.035813 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 207547 | ||
| 6352 | 2026.08.18 00:03:09.123914 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 654 | ||
| 6353 | 2026.08.18 00:03:09.130790 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6354 | 2026.08.18 00:03:09.130805 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6355 | 2026.08.18 00:03:09.134170 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_87_87_0 (state Temporary) | ||
| 6356 | 2026.08.18 00:03:09.136754 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_87_87_0 to 202608_88_88_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6357 | 2026.08.18 00:03:09.137534 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 654 | ||
| 6358 | 2026.08.18 00:03:15.331591 [ 432153 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6359 | 2026.08.18 00:03:15.331619 [ 432153 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6360 | 2026.08.18 00:03:15.535888 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 209801 | ||
| 6361 | 2026.08.18 00:03:15.536964 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6362 | 2026.08.18 00:03:15.536977 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6363 | 2026.08.18 00:03:15.537459 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_88_88_0 (state Temporary) | ||
| 6364 | 2026.08.18 00:03:15.537593 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_88_88_0 to 202608_89_89_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6365 | 2026.08.18 00:03:15.537686 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 209801 | ||
| 6366 | 2026.08.18 00:03:16.637700 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 662 | ||
| 6367 | 2026.08.18 00:03:16.645391 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6368 | 2026.08.18 00:03:16.645406 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6369 | 2026.08.18 00:03:16.648863 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_88_88_0 (state Temporary) | ||
| 6370 | 2026.08.18 00:03:16.651365 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_88_88_0 to 202608_89_89_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6371 | 2026.08.18 00:03:16.652148 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 662 | ||
| 6372 | 2026.08.18 00:03:22.516069 [ 432236 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Found 6 old parts to remove. Parts: [202608_1_21_4, 202608_22_22_0, 202608_23_23_0, 202608_24_24_0, 202608_25_25_0, 202608_26_26_0] | ||
| 6373 | 2026.08.18 00:03:22.516082 [ 432236 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from filesystem (serially): Parts: [202608_1_21_4, 202608_22_22_0, 202608_23_23_0, 202608_24_24_0, 202608_25_25_0, 202608_26_26_0] | ||
| 6374 | 2026.08.18 00:03:22.516764 [ 432236 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from memory: Parts: [202608_1_21_4, 202608_22_22_0, 202608_23_23_0, 202608_24_24_0, 202608_25_25_0, 202608_26_26_0] | ||
| 6375 | 2026.08.18 00:03:23.037950 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 212377 | ||
| 6376 | 2026.08.18 00:03:23.038980 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6377 | 2026.08.18 00:03:23.038992 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6378 | 2026.08.18 00:03:23.039456 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_89_89_0 (state Temporary) | ||
| 6379 | 2026.08.18 00:03:23.039581 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_89_89_0 to 202608_90_90_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6380 | 2026.08.18 00:03:23.039657 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 212377 | ||
| 6381 | 2026.08.18 00:03:24.023481 [ 432237 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Found 6 old parts to remove. Parts: [202608_1_21_4, 202608_22_22_0, 202608_23_23_0, 202608_24_24_0, 202608_25_25_0, 202608_26_26_0] | ||
| 6382 | 2026.08.18 00:03:24.023489 [ 432237 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from filesystem (serially): Parts: [202608_1_21_4, 202608_22_22_0, 202608_23_23_0, 202608_24_24_0, 202608_25_25_0, 202608_26_26_0] | ||
| 6383 | 2026.08.18 00:03:24.024192 [ 432237 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from memory: Parts: [202608_1_21_4, 202608_22_22_0, 202608_23_23_0, 202608_24_24_0, 202608_25_25_0, 202608_26_26_0] | ||
| 6384 | 2026.08.18 00:03:24.152323 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 669 | ||
| 6385 | 2026.08.18 00:03:24.160836 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6386 | 2026.08.18 00:03:24.160851 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6387 | 2026.08.18 00:03:24.164351 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_89_89_0 (state Temporary) | ||
| 6388 | 2026.08.18 00:03:24.166930 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_89_89_0 to 202608_90_90_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6389 | 2026.08.18 00:03:24.167720 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 669 | ||
| 6390 | 2026.08.18 00:03:25.000273 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 566.59 MiB, peak 594.12 MiB, free memory in arenas 102.73 MiB, will set to 565.05 MiB (RSS), difference: -1.54 MiB | ||
| 6391 | 2026.08.18 00:03:30.331714 [ 432063 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6392 | 2026.08.18 00:03:30.331738 [ 432063 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6393 | 2026.08.18 00:03:30.539846 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 214631 | ||
| 6394 | 2026.08.18 00:03:30.540887 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6395 | 2026.08.18 00:03:30.540898 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6396 | 2026.08.18 00:03:30.541381 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_90_90_0 (state Temporary) | ||
| 6397 | 2026.08.18 00:03:30.541558 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_90_90_0 to 202608_91_91_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6398 | 2026.08.18 00:03:30.541640 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 214631 | ||
| 6399 | 2026.08.18 00:03:30.541648 [ 432103 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_86_17 to 202608_91_91_0 | ||
| 6400 | 2026.08.18 00:03:30.541673 [ 432103 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6401 | 2026.08.18 00:03:30.541679 [ 432103 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6402 | 2026.08.18 00:03:30.541772 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_86_17 to 202608_91_91_0 into Compact with storage Full | ||
| 6403 | 2026.08.18 00:03:30.541826 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6404 | 2026.08.18 00:03:30.541841 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 27 marks from part 202608_1_86_17, total 205077 rows starting from the beginning of the part | ||
| 6405 | 2026.08.18 00:03:30.541880 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_87_87_0, total 2254 rows starting from the beginning of the part | ||
| 6406 | 2026.08.18 00:03:30.541897 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_88_88_0, total 2576 rows starting from the beginning of the part | ||
| 6407 | 2026.08.18 00:03:30.541915 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_89_89_0, total 2254 rows starting from the beginning of the part | ||
| 6408 | 2026.08.18 00:03:30.541931 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_90_90_0, total 2576 rows starting from the beginning of the part | ||
| 6409 | 2026.08.18 00:03:30.541947 [ 432211 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_91_91_0, total 2254 rows starting from the beginning of the part | ||
| 6410 | 2026.08.18 00:03:30.555712 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 216991 rows, containing 4 columns (4 merged, 0 gathered) in 0.01397544 sec., 15526595.227055462 rows/sec., 222.92 MiB/sec. | ||
| 6411 | 2026.08.18 00:03:30.556052 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Trace> MergedBlockOutputStream: filled checksums 202608_1_91_18 (state Temporary) | ||
| 6412 | 2026.08.18 00:03:30.556230 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_91_18 to 202608_1_91_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6413 | 2026.08.18 00:03:30.556261 [ 432212 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_91_18} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_86_17, 202608_91_91_0] -> 202608_1_91_18 | ||
| 6414 | 2026.08.18 00:03:30.556331 [ 432212 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.78 MiB. | ||
| 6415 | 2026.08.18 00:03:31.000395 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 565.08 MiB, peak 594.12 MiB, free memory in arenas 85.14 MiB, will set to 559.04 MiB (RSS), difference: -6.04 MiB | ||
| 6416 | 2026.08.18 00:03:31.667807 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 677 | ||
| 6417 | 2026.08.18 00:03:31.675181 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6418 | 2026.08.18 00:03:31.675194 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6419 | 2026.08.18 00:03:31.678524 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_90_90_0 (state Temporary) | ||
| 6420 | 2026.08.18 00:03:31.681040 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_90_90_0 to 202608_91_91_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6421 | 2026.08.18 00:03:31.681171 [ 432023 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_86_17 to 202608_91_91_0 | ||
| 6422 | 2026.08.18 00:03:31.681183 [ 432023 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6423 | 2026.08.18 00:03:31.681188 [ 432023 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6424 | 2026.08.18 00:03:31.681253 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_86_17 to 202608_91_91_0 into Compact with storage Full | ||
| 6425 | 2026.08.18 00:03:31.681844 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 677 | ||
| 6426 | 2026.08.18 00:03:31.685030 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6427 | 2026.08.18 00:03:31.685414 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_86_17, total 647 rows starting from the beginning of the part | ||
| 6428 | 2026.08.18 00:03:31.687325 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_87_87_0, total 8 rows starting from the beginning of the part | ||
| 6429 | 2026.08.18 00:03:31.689149 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_88_88_0, total 7 rows starting from the beginning of the part | ||
| 6430 | 2026.08.18 00:03:31.691029 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_89_89_0, total 8 rows starting from the beginning of the part | ||
| 6431 | 2026.08.18 00:03:31.692841 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_90_90_0, total 7 rows starting from the beginning of the part | ||
| 6432 | 2026.08.18 00:03:31.694659 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_91_91_0, total 8 rows starting from the beginning of the part | ||
| 6433 | 2026.08.18 00:03:31.723755 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 685 rows, containing 679 columns (679 merged, 0 gathered) in 0.042529781 sec., 16106.360858053795 rows/sec., 9.45 MiB/sec. | ||
| 6434 | 2026.08.18 00:03:31.724741 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Trace> MergedBlockOutputStream: filled checksums 202608_1_91_18 (state Temporary) | ||
| 6435 | 2026.08.18 00:03:31.728187 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_91_18 to 202608_1_91_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6436 | 2026.08.18 00:03:31.728225 [ 432199 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_91_18} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_86_17, 202608_91_91_0] -> 202608_1_91_18 | ||
| 6437 | 2026.08.18 00:03:31.728654 [ 432199 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.63 MiB. | ||
| 6438 | 2026.08.18 00:03:32.000337 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 560.50 MiB, peak 594.12 MiB, free memory in arenas 110.02 MiB, will set to 566.70 MiB (RSS), difference: 6.19 MiB | ||
| 6439 | 2026.08.18 00:03:32.060062 [ 432107 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73347504128, merge_with_ttl_allowedtrue) | ||
| 6440 | 2026.08.18 00:03:32.060071 [ 432107 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 6441 | 2026.08.18 00:03:38.041818 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 217207 | ||
| 6442 | 2026.08.18 00:03:38.042905 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6443 | 2026.08.18 00:03:38.042917 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6444 | 2026.08.18 00:03:38.043389 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_91_91_0 (state Temporary) | ||
| 6445 | 2026.08.18 00:03:38.043525 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_91_91_0 to 202608_92_92_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6446 | 2026.08.18 00:03:38.043609 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 217207 | ||
| 6447 | 2026.08.18 00:03:39.182033 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 684 | ||
| 6448 | 2026.08.18 00:03:39.190010 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6449 | 2026.08.18 00:03:39.190026 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6450 | 2026.08.18 00:03:39.193478 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_91_91_0 (state Temporary) | ||
| 6451 | 2026.08.18 00:03:39.196043 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_91_91_0 to 202608_92_92_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6452 | 2026.08.18 00:03:39.196829 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 684 | ||
| 6453 | 2026.08.18 00:03:45.331929 [ 432118 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6454 | 2026.08.18 00:03:45.331955 [ 432118 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6455 | 2026.08.18 00:03:45.543846 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 219461 | ||
| 6456 | 2026.08.18 00:03:45.544835 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6457 | 2026.08.18 00:03:45.544848 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6458 | 2026.08.18 00:03:45.545279 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_92_92_0 (state Temporary) | ||
| 6459 | 2026.08.18 00:03:45.545414 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_92_92_0 to 202608_93_93_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6460 | 2026.08.18 00:03:45.545501 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 219461 | ||
| 6461 | 2026.08.18 00:03:46.696925 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 692 | ||
| 6462 | 2026.08.18 00:03:46.709240 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6463 | 2026.08.18 00:03:46.709259 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6464 | 2026.08.18 00:03:46.713495 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_92_92_0 (state Temporary) | ||
| 6465 | 2026.08.18 00:03:46.716473 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_92_92_0 to 202608_93_93_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6466 | 2026.08.18 00:03:46.717288 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 692 | ||
| 6467 | 2026.08.18 00:03:49.479442 [ 432139 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 6468 | 2026.08.18 00:03:49.479457 [ 432084 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346607104, merge_with_ttl_allowedtrue) | ||
| 6469 | 2026.08.18 00:03:49.479471 [ 432084 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 6470 | 2026.08.18 00:03:53.045662 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 222037 | ||
| 6471 | 2026.08.18 00:03:53.046809 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6472 | 2026.08.18 00:03:53.046823 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6473 | 2026.08.18 00:03:53.047321 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_93_93_0 (state Temporary) | ||
| 6474 | 2026.08.18 00:03:53.047451 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_93_93_0 to 202608_94_94_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6475 | 2026.08.18 00:03:53.047539 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 222037 | ||
| 6476 | 2026.08.18 00:03:54.110177 [ 432146 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346361344, merge_with_ttl_allowedtrue) | ||
| 6477 | 2026.08.18 00:03:54.110192 [ 432146 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 6478 | 2026.08.18 00:03:54.194980 [ 432149 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346361344, merge_with_ttl_allowedtrue) | ||
| 6479 | 2026.08.18 00:03:54.194989 [ 432149 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 6480 | 2026.08.18 00:03:54.217459 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 699 | ||
| 6481 | 2026.08.18 00:03:54.226244 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6482 | 2026.08.18 00:03:54.226258 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6483 | 2026.08.18 00:03:54.229688 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_93_93_0 (state Temporary) | ||
| 6484 | 2026.08.18 00:03:54.232299 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_93_93_0 to 202608_94_94_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6485 | 2026.08.18 00:03:54.233064 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 699 | ||
| 6486 | 2026.08.18 00:04:00.332136 [ 432132 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6487 | 2026.08.18 00:04:00.332164 [ 432132 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6488 | 2026.08.18 00:04:00.547723 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 224291 | ||
| 6489 | 2026.08.18 00:04:00.548836 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6490 | 2026.08.18 00:04:00.548850 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6491 | 2026.08.18 00:04:00.549333 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_94_94_0 (state Temporary) | ||
| 6492 | 2026.08.18 00:04:00.549462 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_94_94_0 to 202608_95_95_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6493 | 2026.08.18 00:04:00.549545 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 224291 | ||
| 6494 | 2026.08.18 00:04:01.000313 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 569.08 MiB, peak 594.12 MiB, free memory in arenas 110.09 MiB, will set to 568.01 MiB (RSS), difference: -1.07 MiB | ||
| 6495 | 2026.08.18 00:04:01.091786 [ 432236 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Found 6 old parts to remove. Parts: [202608_1_26_5, 202608_27_27_0, 202608_28_28_0, 202608_29_29_0, 202608_30_30_0, 202608_31_31_0] | ||
| 6496 | 2026.08.18 00:04:01.091796 [ 432236 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from filesystem (serially): Parts: [202608_1_26_5, 202608_27_27_0, 202608_28_28_0, 202608_29_29_0, 202608_30_30_0, 202608_31_31_0] | ||
| 6497 | 2026.08.18 00:04:01.092474 [ 432236 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from memory: Parts: [202608_1_26_5, 202608_27_27_0, 202608_28_28_0, 202608_29_29_0, 202608_30_30_0, 202608_31_31_0] | ||
| 6498 | 2026.08.18 00:04:01.338875 [ 432232 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Found 6 old parts to remove. Parts: [202608_1_26_5, 202608_27_27_0, 202608_28_28_0, 202608_29_29_0, 202608_30_30_0, 202608_31_31_0] | ||
| 6499 | 2026.08.18 00:04:01.338884 [ 432232 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from filesystem (serially): Parts: [202608_1_26_5, 202608_27_27_0, 202608_28_28_0, 202608_29_29_0, 202608_30_30_0, 202608_31_31_0] | ||
| 6500 | 2026.08.18 00:04:01.339672 [ 432232 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from memory: Parts: [202608_1_26_5, 202608_27_27_0, 202608_28_28_0, 202608_29_29_0, 202608_30_30_0, 202608_31_31_0] | ||
| 6501 | 2026.08.18 00:04:01.733240 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 707 | ||
| 6502 | 2026.08.18 00:04:01.741747 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6503 | 2026.08.18 00:04:01.741762 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6504 | 2026.08.18 00:04:01.745349 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_94_94_0 (state Temporary) | ||
| 6505 | 2026.08.18 00:04:01.748023 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_94_94_0 to 202608_95_95_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6506 | 2026.08.18 00:04:01.748808 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 707 | ||
| 6507 | 2026.08.18 00:04:07.000274 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 568.72 MiB, peak 594.12 MiB, free memory in arenas 100.49 MiB, will set to 564.72 MiB (RSS), difference: -4.00 MiB | ||
| 6508 | 2026.08.18 00:04:08.049716 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 226867 | ||
| 6509 | 2026.08.18 00:04:08.050839 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6510 | 2026.08.18 00:04:08.050852 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6511 | 2026.08.18 00:04:08.051238 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_95_95_0 (state Temporary) | ||
| 6512 | 2026.08.18 00:04:08.051471 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_95_95_0 to 202608_96_96_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6513 | 2026.08.18 00:04:08.051559 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 226867 | ||
| 6514 | 2026.08.18 00:04:08.051560 [ 432032 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_91_18 to 202608_96_96_0 | ||
| 6515 | 2026.08.18 00:04:08.051578 [ 432032 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6516 | 2026.08.18 00:04:08.051583 [ 432032 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6517 | 2026.08.18 00:04:08.051672 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_91_18 to 202608_96_96_0 into Compact with storage Full | ||
| 6518 | 2026.08.18 00:04:08.051750 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6519 | 2026.08.18 00:04:08.051770 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 28 marks from part 202608_1_91_18, total 216991 rows starting from the beginning of the part | ||
| 6520 | 2026.08.18 00:04:08.051812 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_92_92_0, total 2576 rows starting from the beginning of the part | ||
| 6521 | 2026.08.18 00:04:08.051832 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_93_93_0, total 2254 rows starting from the beginning of the part | ||
| 6522 | 2026.08.18 00:04:08.051850 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_94_94_0, total 2576 rows starting from the beginning of the part | ||
| 6523 | 2026.08.18 00:04:08.051868 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_95_95_0, total 2254 rows starting from the beginning of the part | ||
| 6524 | 2026.08.18 00:04:08.051922 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_96_96_0, total 2576 rows starting from the beginning of the part | ||
| 6525 | 2026.08.18 00:04:08.067182 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 229227 rows, containing 4 columns (4 merged, 0 gathered) in 0.015555971 sec., 14735627.882052492 rows/sec., 211.54 MiB/sec. | ||
| 6526 | 2026.08.18 00:04:08.067407 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Trace> MergedBlockOutputStream: filled checksums 202608_1_96_19 (state Temporary) | ||
| 6527 | 2026.08.18 00:04:08.067552 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_96_19 to 202608_1_96_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6528 | 2026.08.18 00:04:08.067576 [ 432198 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_96_19} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_91_18, 202608_96_96_0] -> 202608_1_96_19 | ||
| 6529 | 2026.08.18 00:04:08.067620 [ 432198 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 6530 | 2026.08.18 00:04:09.000311 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 564.46 MiB, peak 594.12 MiB, free memory in arenas 112.97 MiB, will set to 556.31 MiB (RSS), difference: -8.15 MiB | ||
| 6531 | 2026.08.18 00:04:09.248980 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 714 | ||
| 6532 | 2026.08.18 00:04:09.256345 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6533 | 2026.08.18 00:04:09.256360 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6534 | 2026.08.18 00:04:09.259727 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_95_95_0 (state Temporary) | ||
| 6535 | 2026.08.18 00:04:09.262336 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_95_95_0 to 202608_96_96_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6536 | 2026.08.18 00:04:09.262481 [ 432092 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_91_18 to 202608_96_96_0 | ||
| 6537 | 2026.08.18 00:04:09.262495 [ 432092 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6538 | 2026.08.18 00:04:09.262499 [ 432092 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6539 | 2026.08.18 00:04:09.262594 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_91_18 to 202608_96_96_0 into Compact with storage Full | ||
| 6540 | 2026.08.18 00:04:09.263168 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 714 | ||
| 6541 | 2026.08.18 00:04:09.266249 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6542 | 2026.08.18 00:04:09.266663 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_91_18, total 685 rows starting from the beginning of the part | ||
| 6543 | 2026.08.18 00:04:09.268616 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_92_92_0, total 7 rows starting from the beginning of the part | ||
| 6544 | 2026.08.18 00:04:09.270536 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_93_93_0, total 8 rows starting from the beginning of the part | ||
| 6545 | 2026.08.18 00:04:09.272481 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_94_94_0, total 7 rows starting from the beginning of the part | ||
| 6546 | 2026.08.18 00:04:09.274341 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_95_95_0, total 8 rows starting from the beginning of the part | ||
| 6547 | 2026.08.18 00:04:09.276251 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_96_96_0, total 7 rows starting from the beginning of the part | ||
| 6548 | 2026.08.18 00:04:09.305516 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 722 rows, containing 679 columns (679 merged, 0 gathered) in 0.042961011 sec., 16805.93596831322 rows/sec., 9.76 MiB/sec. | ||
| 6549 | 2026.08.18 00:04:09.306515 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Trace> MergedBlockOutputStream: filled checksums 202608_1_96_19 (state Temporary) | ||
| 6550 | 2026.08.18 00:04:09.309899 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_96_19 to 202608_1_96_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6551 | 2026.08.18 00:04:09.309943 [ 432225 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_96_19} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_91_18, 202608_96_96_0] -> 202608_1_96_19 | ||
| 6552 | 2026.08.18 00:04:09.310412 [ 432225 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.63 MiB. | ||
| 6553 | 2026.08.18 00:04:10.000487 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 557.77 MiB, peak 594.12 MiB, free memory in arenas 139.97 MiB, will set to 577.20 MiB (RSS), difference: 19.43 MiB | ||
| 6554 | 2026.08.18 00:04:15.332269 [ 432058 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6555 | 2026.08.18 00:04:15.332296 [ 432058 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6556 | 2026.08.18 00:04:15.551733 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 229121 | ||
| 6557 | 2026.08.18 00:04:15.552841 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6558 | 2026.08.18 00:04:15.552859 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6559 | 2026.08.18 00:04:15.553296 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_96_96_0 (state Temporary) | ||
| 6560 | 2026.08.18 00:04:15.553425 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_96_96_0 to 202608_97_97_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6561 | 2026.08.18 00:04:15.553502 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 229121 | ||
| 6562 | 2026.08.18 00:04:16.000889 [ 432270 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 8.181e-05 sec. | ||
| 6563 | 2026.08.18 00:04:16.763336 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 722 | ||
| 6564 | 2026.08.18 00:04:16.770597 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6565 | 2026.08.18 00:04:16.770612 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6566 | 2026.08.18 00:04:16.773963 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_96_96_0 (state Temporary) | ||
| 6567 | 2026.08.18 00:04:16.776517 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_96_96_0 to 202608_97_97_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6568 | 2026.08.18 00:04:16.777299 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 722 | ||
| 6569 | 2026.08.18 00:04:23.053680 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 231697 | ||
| 6570 | 2026.08.18 00:04:23.054868 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6571 | 2026.08.18 00:04:23.054881 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6572 | 2026.08.18 00:04:23.055351 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_97_97_0 (state Temporary) | ||
| 6573 | 2026.08.18 00:04:23.055496 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_97_97_0 to 202608_98_98_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6574 | 2026.08.18 00:04:23.055586 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 231697 | ||
| 6575 | 2026.08.18 00:04:24.277481 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 729 | ||
| 6576 | 2026.08.18 00:04:24.285893 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6577 | 2026.08.18 00:04:24.285908 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6578 | 2026.08.18 00:04:24.289370 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_97_97_0 (state Temporary) | ||
| 6579 | 2026.08.18 00:04:24.292014 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_97_97_0 to 202608_98_98_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6580 | 2026.08.18 00:04:24.292782 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 729 | ||
| 6581 | 2026.08.18 00:04:30.332478 [ 432152 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6582 | 2026.08.18 00:04:30.332503 [ 432152 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6583 | 2026.08.18 00:04:30.555781 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 233951 | ||
| 6584 | 2026.08.18 00:04:30.556838 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6585 | 2026.08.18 00:04:30.556851 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6586 | 2026.08.18 00:04:30.557341 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_98_98_0 (state Temporary) | ||
| 6587 | 2026.08.18 00:04:30.557469 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_98_98_0 to 202608_99_99_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6588 | 2026.08.18 00:04:30.557550 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 233951 | ||
| 6589 | 2026.08.18 00:04:31.792962 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 737 | ||
| 6590 | 2026.08.18 00:04:31.801113 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6591 | 2026.08.18 00:04:31.801129 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6592 | 2026.08.18 00:04:31.804593 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_98_98_0 (state Temporary) | ||
| 6593 | 2026.08.18 00:04:31.807208 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_98_98_0 to 202608_99_99_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6594 | 2026.08.18 00:04:31.807986 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 737 | ||
| 6595 | 2026.08.18 00:04:32.060258 [ 432130 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345953792, merge_with_ttl_allowedtrue) | ||
| 6596 | 2026.08.18 00:04:32.060271 [ 432130 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 6597 | 2026.08.18 00:04:37.765960 [ 432236 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Found 6 old parts to remove. Parts: [202608_1_31_6, 202608_32_32_0, 202608_33_33_0, 202608_34_34_0, 202608_35_35_0, 202608_36_36_0] | ||
| 6598 | 2026.08.18 00:04:37.765972 [ 432236 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from filesystem (serially): Parts: [202608_1_31_6, 202608_32_32_0, 202608_33_33_0, 202608_34_34_0, 202608_35_35_0, 202608_36_36_0] | ||
| 6599 | 2026.08.18 00:04:37.766684 [ 432236 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from memory: Parts: [202608_1_31_6, 202608_32_32_0, 202608_33_33_0, 202608_34_34_0, 202608_35_35_0, 202608_36_36_0] | ||
| 6600 | 2026.08.18 00:04:38.057710 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 236527 | ||
| 6601 | 2026.08.18 00:04:38.058857 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6602 | 2026.08.18 00:04:38.058870 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6603 | 2026.08.18 00:04:38.059323 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_99_99_0 (state Temporary) | ||
| 6604 | 2026.08.18 00:04:38.059448 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_99_99_0 to 202608_100_100_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6605 | 2026.08.18 00:04:38.059521 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 236527 | ||
| 6606 | 2026.08.18 00:04:38.467359 [ 432232 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Found 6 old parts to remove. Parts: [202608_1_31_6, 202608_32_32_0, 202608_33_33_0, 202608_34_34_0, 202608_35_35_0, 202608_36_36_0] | ||
| 6607 | 2026.08.18 00:04:38.467367 [ 432232 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from filesystem (serially): Parts: [202608_1_31_6, 202608_32_32_0, 202608_33_33_0, 202608_34_34_0, 202608_35_35_0, 202608_36_36_0] | ||
| 6608 | 2026.08.18 00:04:38.468117 [ 432232 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from memory: Parts: [202608_1_31_6, 202608_32_32_0, 202608_33_33_0, 202608_34_34_0, 202608_35_35_0, 202608_36_36_0] | ||
| 6609 | 2026.08.18 00:04:39.000322 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 575.33 MiB, peak 594.12 MiB, free memory in arenas 130.48 MiB, will set to 573.63 MiB (RSS), difference: -1.71 MiB | ||
| 6610 | 2026.08.18 00:04:39.308160 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 744 | ||
| 6611 | 2026.08.18 00:04:39.316667 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6612 | 2026.08.18 00:04:39.316683 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6613 | 2026.08.18 00:04:39.320300 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_99_99_0 (state Temporary) | ||
| 6614 | 2026.08.18 00:04:39.322975 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_99_99_0 to 202608_100_100_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6615 | 2026.08.18 00:04:39.323772 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 744 | ||
| 6616 | 2026.08.18 00:04:40.000282 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 574.33 MiB, peak 594.12 MiB, free memory in arenas 139.71 MiB, will set to 576.75 MiB (RSS), difference: 2.41 MiB | ||
| 6617 | 2026.08.18 00:04:45.332693 [ 432056 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6618 | 2026.08.18 00:04:45.332722 [ 432056 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6619 | 2026.08.18 00:04:45.559697 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 238781 | ||
| 6620 | 2026.08.18 00:04:45.560706 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6621 | 2026.08.18 00:04:45.560718 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6622 | 2026.08.18 00:04:45.561082 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_100_100_0 (state Temporary) | ||
| 6623 | 2026.08.18 00:04:45.561211 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_100_100_0 to 202608_101_101_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6624 | 2026.08.18 00:04:45.561292 [ 432022 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_96_19 to 202608_101_101_0 | ||
| 6625 | 2026.08.18 00:04:45.561299 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 238781 | ||
| 6626 | 2026.08.18 00:04:45.561306 [ 432022 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6627 | 2026.08.18 00:04:45.561311 [ 432022 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6628 | 2026.08.18 00:04:45.561413 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_96_19 to 202608_101_101_0 into Compact with storage Full | ||
| 6629 | 2026.08.18 00:04:45.561493 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6630 | 2026.08.18 00:04:45.561513 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 29 marks from part 202608_1_96_19, total 229227 rows starting from the beginning of the part | ||
| 6631 | 2026.08.18 00:04:45.561555 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_97_97_0, total 2254 rows starting from the beginning of the part | ||
| 6632 | 2026.08.18 00:04:45.561577 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_98_98_0, total 2576 rows starting from the beginning of the part | ||
| 6633 | 2026.08.18 00:04:45.561596 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_99_99_0, total 2254 rows starting from the beginning of the part | ||
| 6634 | 2026.08.18 00:04:45.561613 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_100_100_0, total 2576 rows starting from the beginning of the part | ||
| 6635 | 2026.08.18 00:04:45.561641 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_101_101_0, total 2254 rows starting from the beginning of the part | ||
| 6636 | 2026.08.18 00:04:45.577034 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 241141 rows, containing 4 columns (4 merged, 0 gathered) in 0.015667911 sec., 15390756.304398207 rows/sec., 220.89 MiB/sec. | ||
| 6637 | 2026.08.18 00:04:45.577204 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Trace> MergedBlockOutputStream: filled checksums 202608_1_101_20 (state Temporary) | ||
| 6638 | 2026.08.18 00:04:45.577338 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_101_20 to 202608_1_101_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6639 | 2026.08.18 00:04:45.577363 [ 432216 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_101_20} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_96_19, 202608_101_101_0] -> 202608_1_101_20 | ||
| 6640 | 2026.08.18 00:04:45.577407 [ 432216 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 6641 | 2026.08.18 00:04:46.000298 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 576.77 MiB, peak 594.12 MiB, free memory in arenas 152.64 MiB, will set to 578.03 MiB (RSS), difference: 1.26 MiB | ||
| 6642 | 2026.08.18 00:04:46.823853 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 752 | ||
| 6643 | 2026.08.18 00:04:46.831107 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6644 | 2026.08.18 00:04:46.831122 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6645 | 2026.08.18 00:04:46.834381 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_100_100_0 (state Temporary) | ||
| 6646 | 2026.08.18 00:04:46.836905 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_100_100_0 to 202608_101_101_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6647 | 2026.08.18 00:04:46.837053 [ 432016 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_96_19 to 202608_101_101_0 | ||
| 6648 | 2026.08.18 00:04:46.837073 [ 432016 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6649 | 2026.08.18 00:04:46.837079 [ 432016 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6650 | 2026.08.18 00:04:46.837149 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_96_19 to 202608_101_101_0 into Compact with storage Full | ||
| 6651 | 2026.08.18 00:04:46.837684 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 752 | ||
| 6652 | 2026.08.18 00:04:46.841218 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6653 | 2026.08.18 00:04:46.841575 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_96_19, total 722 rows starting from the beginning of the part | ||
| 6654 | 2026.08.18 00:04:46.843422 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_97_97_0, total 8 rows starting from the beginning of the part | ||
| 6655 | 2026.08.18 00:04:46.845212 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_98_98_0, total 7 rows starting from the beginning of the part | ||
| 6656 | 2026.08.18 00:04:46.847054 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_99_99_0, total 8 rows starting from the beginning of the part | ||
| 6657 | 2026.08.18 00:04:46.848805 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_100_100_0, total 7 rows starting from the beginning of the part | ||
| 6658 | 2026.08.18 00:04:46.850596 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_101_101_0, total 8 rows starting from the beginning of the part | ||
| 6659 | 2026.08.18 00:04:46.878817 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 760 rows, containing 679 columns (679 merged, 0 gathered) in 0.04169257 sec., 18228.667601925237 rows/sec., 10.53 MiB/sec. | ||
| 6660 | 2026.08.18 00:04:46.879806 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Trace> MergedBlockOutputStream: filled checksums 202608_1_101_20 (state Temporary) | ||
| 6661 | 2026.08.18 00:04:46.883183 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_101_20 to 202608_1_101_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6662 | 2026.08.18 00:04:46.883227 [ 432224 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_101_20} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_96_19, 202608_101_101_0] -> 202608_1_101_20 | ||
| 6663 | 2026.08.18 00:04:46.883812 [ 432224 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.63 MiB. | ||
| 6664 | 2026.08.18 00:04:47.000287 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 579.49 MiB, peak 597.37 MiB, free memory in arenas 160.86 MiB, will set to 583.20 MiB (RSS), difference: 3.71 MiB | ||
| 6665 | 2026.08.18 00:04:49.479519 [ 432026 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 6666 | 2026.08.18 00:04:49.479537 [ 432037 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346850816, merge_with_ttl_allowedtrue) | ||
| 6667 | 2026.08.18 00:04:49.479557 [ 432037 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 6668 | 2026.08.18 00:04:53.061572 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 241357 | ||
| 6669 | 2026.08.18 00:04:53.062786 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6670 | 2026.08.18 00:04:53.062798 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6671 | 2026.08.18 00:04:53.063293 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_101_101_0 (state Temporary) | ||
| 6672 | 2026.08.18 00:04:53.063424 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_101_101_0 to 202608_102_102_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6673 | 2026.08.18 00:04:53.063509 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 241357 | ||
| 6674 | 2026.08.18 00:04:54.110358 [ 432129 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346416640, merge_with_ttl_allowedtrue) | ||
| 6675 | 2026.08.18 00:04:54.110378 [ 432129 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 6676 | 2026.08.18 00:04:54.195190 [ 432133 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346416640, merge_with_ttl_allowedtrue) | ||
| 6677 | 2026.08.18 00:04:54.195202 [ 432133 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 6678 | 2026.08.18 00:04:54.337862 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 759 | ||
| 6679 | 2026.08.18 00:04:54.345319 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6680 | 2026.08.18 00:04:54.345333 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6681 | 2026.08.18 00:04:54.348731 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_101_101_0 (state Temporary) | ||
| 6682 | 2026.08.18 00:04:54.351337 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_101_101_0 to 202608_102_102_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6683 | 2026.08.18 00:04:54.352122 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 759 | ||
| 6684 | 2026.08.18 00:05:00.332904 [ 432116 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6685 | 2026.08.18 00:05:00.332933 [ 432116 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6686 | 2026.08.18 00:05:00.563688 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 243611 | ||
| 6687 | 2026.08.18 00:05:00.564706 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6688 | 2026.08.18 00:05:00.564719 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6689 | 2026.08.18 00:05:00.565176 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_102_102_0 (state Temporary) | ||
| 6690 | 2026.08.18 00:05:00.565312 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_102_102_0 to 202608_103_103_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6691 | 2026.08.18 00:05:00.565393 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 243611 | ||
| 6692 | 2026.08.18 00:05:01.852290 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 767 | ||
| 6693 | 2026.08.18 00:05:01.859805 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6694 | 2026.08.18 00:05:01.859819 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6695 | 2026.08.18 00:05:01.863281 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_102_102_0 (state Temporary) | ||
| 6696 | 2026.08.18 00:05:01.865763 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_102_102_0 to 202608_103_103_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6697 | 2026.08.18 00:05:01.866527 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 767 | ||
| 6698 | 2026.08.18 00:05:08.065560 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 246187 | ||
| 6699 | 2026.08.18 00:05:08.066712 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6700 | 2026.08.18 00:05:08.066725 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6701 | 2026.08.18 00:05:08.067212 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_103_103_0 (state Temporary) | ||
| 6702 | 2026.08.18 00:05:08.067342 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_103_103_0 to 202608_104_104_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6703 | 2026.08.18 00:05:08.067425 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 246187 | ||
| 6704 | 2026.08.18 00:05:09.366615 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 775 | ||
| 6705 | 2026.08.18 00:05:09.374664 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6706 | 2026.08.18 00:05:09.374678 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6707 | 2026.08.18 00:05:09.378164 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_103_103_0 (state Temporary) | ||
| 6708 | 2026.08.18 00:05:09.380769 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_103_103_0 to 202608_104_104_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6709 | 2026.08.18 00:05:09.381537 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 775 | ||
| 6710 | 2026.08.18 00:05:15.182412 [ 432237 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Found 6 old parts to remove. Parts: [202608_1_36_7, 202608_37_37_0, 202608_38_38_0, 202608_39_39_0, 202608_40_40_0, 202608_41_41_0] | ||
| 6711 | 2026.08.18 00:05:15.182426 [ 432237 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from filesystem (serially): Parts: [202608_1_36_7, 202608_37_37_0, 202608_38_38_0, 202608_39_39_0, 202608_40_40_0, 202608_41_41_0] | ||
| 6712 | 2026.08.18 00:05:15.183081 [ 432237 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from memory: Parts: [202608_1_36_7, 202608_37_37_0, 202608_38_38_0, 202608_39_39_0, 202608_40_40_0, 202608_41_41_0] | ||
| 6713 | 2026.08.18 00:05:15.333118 [ 432074 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6714 | 2026.08.18 00:05:15.333147 [ 432074 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6715 | 2026.08.18 00:05:15.567513 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 248441 | ||
| 6716 | 2026.08.18 00:05:15.568587 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6717 | 2026.08.18 00:05:15.568600 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6718 | 2026.08.18 00:05:15.569064 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_104_104_0 (state Temporary) | ||
| 6719 | 2026.08.18 00:05:15.569196 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_104_104_0 to 202608_105_105_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6720 | 2026.08.18 00:05:15.569276 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 248441 | ||
| 6721 | 2026.08.18 00:05:16.881708 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 782 | ||
| 6722 | 2026.08.18 00:05:16.889493 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6723 | 2026.08.18 00:05:16.889508 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6724 | 2026.08.18 00:05:16.892962 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_104_104_0 (state Temporary) | ||
| 6725 | 2026.08.18 00:05:16.895538 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_104_104_0 to 202608_105_105_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6726 | 2026.08.18 00:05:16.895762 [ 432243 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Found 6 old parts to remove. Parts: [202608_1_36_7, 202608_37_37_0, 202608_38_38_0, 202608_39_39_0, 202608_40_40_0, 202608_41_41_0] | ||
| 6727 | 2026.08.18 00:05:16.895773 [ 432243 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from filesystem (serially): Parts: [202608_1_36_7, 202608_37_37_0, 202608_38_38_0, 202608_39_39_0, 202608_40_40_0, 202608_41_41_0] | ||
| 6728 | 2026.08.18 00:05:16.896317 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 782 | ||
| 6729 | 2026.08.18 00:05:16.896529 [ 432243 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from memory: Parts: [202608_1_36_7, 202608_37_37_0, 202608_38_38_0, 202608_39_39_0, 202608_40_40_0, 202608_41_41_0] | ||
| 6730 | 2026.08.18 00:05:17.000288 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 581.18 MiB, peak 597.37 MiB, free memory in arenas 142.12 MiB, will set to 569.97 MiB (RSS), difference: -11.21 MiB | ||
| 6731 | 2026.08.18 00:05:23.069440 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 251017 | ||
| 6732 | 2026.08.18 00:05:23.070603 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6733 | 2026.08.18 00:05:23.070616 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6734 | 2026.08.18 00:05:23.071084 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_105_105_0 (state Temporary) | ||
| 6735 | 2026.08.18 00:05:23.071232 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_105_105_0 to 202608_106_106_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6736 | 2026.08.18 00:05:23.071309 [ 432072 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_101_20 to 202608_106_106_0 | ||
| 6737 | 2026.08.18 00:05:23.071312 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 251017 | ||
| 6738 | 2026.08.18 00:05:23.071329 [ 432072 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6739 | 2026.08.18 00:05:23.071334 [ 432072 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6740 | 2026.08.18 00:05:23.071435 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_101_20 to 202608_106_106_0 into Compact with storage Full | ||
| 6741 | 2026.08.18 00:05:23.071512 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6742 | 2026.08.18 00:05:23.071531 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 31 marks from part 202608_1_101_20, total 241141 rows starting from the beginning of the part | ||
| 6743 | 2026.08.18 00:05:23.071572 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_102_102_0, total 2576 rows starting from the beginning of the part | ||
| 6744 | 2026.08.18 00:05:23.071589 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_103_103_0, total 2254 rows starting from the beginning of the part | ||
| 6745 | 2026.08.18 00:05:23.071606 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_104_104_0, total 2576 rows starting from the beginning of the part | ||
| 6746 | 2026.08.18 00:05:23.071622 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_105_105_0, total 2254 rows starting from the beginning of the part | ||
| 6747 | 2026.08.18 00:05:23.071638 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_106_106_0, total 2576 rows starting from the beginning of the part | ||
| 6748 | 2026.08.18 00:05:23.087615 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 253377 rows, containing 4 columns (4 merged, 0 gathered) in 0.016227192 sec., 15614346.585656963 rows/sec., 224.08 MiB/sec. | ||
| 6749 | 2026.08.18 00:05:23.087801 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Trace> MergedBlockOutputStream: filled checksums 202608_1_106_21 (state Temporary) | ||
| 6750 | 2026.08.18 00:05:23.087934 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_106_21 to 202608_1_106_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6751 | 2026.08.18 00:05:23.087957 [ 432210 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_106_21} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_101_20, 202608_106_106_0] -> 202608_1_106_21 | ||
| 6752 | 2026.08.18 00:05:23.088002 [ 432210 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 6753 | 2026.08.18 00:05:24.396482 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 790 | ||
| 6754 | 2026.08.18 00:05:24.404099 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6755 | 2026.08.18 00:05:24.404118 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6756 | 2026.08.18 00:05:24.407455 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_105_105_0 (state Temporary) | ||
| 6757 | 2026.08.18 00:05:24.409854 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_105_105_0 to 202608_106_106_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6758 | 2026.08.18 00:05:24.409991 [ 432058 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_101_20 to 202608_106_106_0 | ||
| 6759 | 2026.08.18 00:05:24.410005 [ 432058 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6760 | 2026.08.18 00:05:24.410009 [ 432058 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6761 | 2026.08.18 00:05:24.410075 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_101_20 to 202608_106_106_0 into Compact with storage Full | ||
| 6762 | 2026.08.18 00:05:24.410669 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 790 | ||
| 6763 | 2026.08.18 00:05:24.413352 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6764 | 2026.08.18 00:05:24.413706 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_101_20, total 760 rows starting from the beginning of the part | ||
| 6765 | 2026.08.18 00:05:24.415689 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_102_102_0, total 7 rows starting from the beginning of the part | ||
| 6766 | 2026.08.18 00:05:24.417496 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_103_103_0, total 8 rows starting from the beginning of the part | ||
| 6767 | 2026.08.18 00:05:24.419427 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_104_104_0, total 8 rows starting from the beginning of the part | ||
| 6768 | 2026.08.18 00:05:24.421261 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_105_105_0, total 7 rows starting from the beginning of the part | ||
| 6769 | 2026.08.18 00:05:24.423136 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_106_106_0, total 8 rows starting from the beginning of the part | ||
| 6770 | 2026.08.18 00:05:24.452061 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 798 rows, containing 679 columns (679 merged, 0 gathered) in 0.04201413 sec., 18993.6100069191 rows/sec., 10.89 MiB/sec. | ||
| 6771 | 2026.08.18 00:05:24.453077 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Trace> MergedBlockOutputStream: filled checksums 202608_1_106_21 (state Temporary) | ||
| 6772 | 2026.08.18 00:05:24.456511 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_106_21 to 202608_1_106_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6773 | 2026.08.18 00:05:24.456552 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_106_21} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_101_20, 202608_106_106_0] -> 202608_1_106_21 | ||
| 6774 | 2026.08.18 00:05:24.456981 [ 432216 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.63 MiB. | ||
| 6775 | 2026.08.18 00:05:25.000272 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 570.86 MiB, peak 597.37 MiB, free memory in arenas 155.45 MiB, will set to 582.93 MiB (RSS), difference: 12.07 MiB | ||
| 6776 | 2026.08.18 00:05:30.333325 [ 432121 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6777 | 2026.08.18 00:05:30.333354 [ 432121 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6778 | 2026.08.18 00:05:30.571421 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 253271 | ||
| 6779 | 2026.08.18 00:05:30.572827 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6780 | 2026.08.18 00:05:30.572844 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6781 | 2026.08.18 00:05:30.573411 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_106_106_0 (state Temporary) | ||
| 6782 | 2026.08.18 00:05:30.573571 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_106_106_0 to 202608_107_107_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6783 | 2026.08.18 00:05:30.573666 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 253271 | ||
| 6784 | 2026.08.18 00:05:31.910845 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 797 | ||
| 6785 | 2026.08.18 00:05:31.918775 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6786 | 2026.08.18 00:05:31.918791 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6787 | 2026.08.18 00:05:31.922239 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_106_106_0 (state Temporary) | ||
| 6788 | 2026.08.18 00:05:31.924871 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_106_106_0 to 202608_107_107_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6789 | 2026.08.18 00:05:31.925645 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 797 | ||
| 6790 | 2026.08.18 00:05:32.060464 [ 432144 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346007040, merge_with_ttl_allowedtrue) | ||
| 6791 | 2026.08.18 00:05:32.060481 [ 432144 ] {} <Trace> oximeter.fields_uuid_local (eaa5beef-895d-41af-915b-bd6fb56613c4): Scheduling next merge selecting task after 60000ms | ||
| 6792 | 2026.08.18 00:05:38.073869 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 255847 | ||
| 6793 | 2026.08.18 00:05:38.075018 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6794 | 2026.08.18 00:05:38.075032 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6795 | 2026.08.18 00:05:38.075520 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_107_107_0 (state Temporary) | ||
| 6796 | 2026.08.18 00:05:38.075653 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_107_107_0 to 202608_108_108_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6797 | 2026.08.18 00:05:38.075735 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 255847 | ||
| 6798 | 2026.08.18 00:05:39.425805 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 805 | ||
| 6799 | 2026.08.18 00:05:39.432976 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6800 | 2026.08.18 00:05:39.432991 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6801 | 2026.08.18 00:05:39.436416 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_107_107_0 (state Temporary) | ||
| 6802 | 2026.08.18 00:05:39.438889 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_107_107_0 to 202608_108_108_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6803 | 2026.08.18 00:05:39.439682 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 805 | ||
| 6804 | 2026.08.18 00:05:45.333535 [ 432125 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6805 | 2026.08.18 00:05:45.333555 [ 432125 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6806 | 2026.08.18 00:05:45.575815 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 258101 | ||
| 6807 | 2026.08.18 00:05:45.576699 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6808 | 2026.08.18 00:05:45.576710 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6809 | 2026.08.18 00:05:45.577028 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_108_108_0 (state Temporary) | ||
| 6810 | 2026.08.18 00:05:45.577147 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_108_108_0 to 202608_109_109_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6811 | 2026.08.18 00:05:45.577219 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 258101 | ||
| 6812 | 2026.08.18 00:05:46.939860 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 812 | ||
| 6813 | 2026.08.18 00:05:46.947814 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6814 | 2026.08.18 00:05:46.947829 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6815 | 2026.08.18 00:05:46.951206 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_108_108_0 (state Temporary) | ||
| 6816 | 2026.08.18 00:05:46.953722 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_108_108_0 to 202608_109_109_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6817 | 2026.08.18 00:05:46.954505 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 812 | ||
| 6818 | 2026.08.18 00:05:49.479612 [ 432074 ] {} <Trace> oximeter.version (157c979f-eec8-4d9d-8340-ef2e89475af4): Scheduling next merge selecting task after 60000ms | ||
| 6819 | 2026.08.18 00:05:49.479624 [ 432103 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73345294336, merge_with_ttl_allowedtrue) | ||
| 6820 | 2026.08.18 00:05:49.479633 [ 432103 ] {} <Trace> oximeter.timeseries_schema_local (05a0062d-b532-435a-91d7-b9da99374f1d): Scheduling next merge selecting task after 60000ms | ||
| 6821 | 2026.08.18 00:05:53.034443 [ 432234 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Found 6 old parts to remove. Parts: [202608_1_41_8, 202608_42_42_0, 202608_43_43_0, 202608_44_44_0, 202608_45_45_0, 202608_46_46_0] | ||
| 6822 | 2026.08.18 00:05:53.034454 [ 432234 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from filesystem (serially): Parts: [202608_1_41_8, 202608_42_42_0, 202608_43_43_0, 202608_44_44_0, 202608_45_45_0, 202608_46_46_0] | ||
| 6823 | 2026.08.18 00:05:53.035170 [ 432234 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Removing 6 parts from memory: Parts: [202608_1_41_8, 202608_42_42_0, 202608_43_43_0, 202608_44_44_0, 202608_45_45_0, 202608_46_46_0] | ||
| 6824 | 2026.08.18 00:05:53.077308 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 260677 | ||
| 6825 | 2026.08.18 00:05:53.078419 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6826 | 2026.08.18 00:05:53.078436 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6827 | 2026.08.18 00:05:53.078779 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_109_109_0 (state Temporary) | ||
| 6828 | 2026.08.18 00:05:53.078913 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_109_109_0 to 202608_110_110_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6829 | 2026.08.18 00:05:53.078994 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 260677 | ||
| 6830 | 2026.08.18 00:05:53.848096 [ 432243 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Found 6 old parts to remove. Parts: [202608_1_41_8, 202608_42_42_0, 202608_43_43_0, 202608_44_44_0, 202608_45_45_0, 202608_46_46_0] | ||
| 6831 | 2026.08.18 00:05:53.848115 [ 432243 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from filesystem (serially): Parts: [202608_1_41_8, 202608_42_42_0, 202608_43_43_0, 202608_44_44_0, 202608_45_45_0, 202608_46_46_0] | ||
| 6832 | 2026.08.18 00:05:53.852224 [ 432243 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Removing 6 parts from memory: Parts: [202608_1_41_8, 202608_42_42_0, 202608_43_43_0, 202608_44_44_0, 202608_45_45_0, 202608_46_46_0] | ||
| 6833 | 2026.08.18 00:05:54.000291 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 580.46 MiB, peak 597.37 MiB, free memory in arenas 129.46 MiB, will set to 572.98 MiB (RSS), difference: -7.48 MiB | ||
| 6834 | 2026.08.18 00:05:54.110570 [ 432104 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346369536, merge_with_ttl_allowedtrue) | ||
| 6835 | 2026.08.18 00:05:54.110581 [ 432104 ] {} <Trace> oximeter.measurements_cumulativef64_local (24dea9d7-064d-49c2-9599-9587402bf78f): Scheduling next merge selecting task after 60000ms | ||
| 6836 | 2026.08.18 00:05:54.195398 [ 432088 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae) (MergerMutator): Checked 1 partitions, found 0 partitions with parts that may be merged: [](max_total_size_to_merge=73346369536, merge_with_ttl_allowedtrue) | ||
| 6837 | 2026.08.18 00:05:54.195412 [ 432088 ] {} <Trace> oximeter.fields_i64_local (2d79911c-8882-4309-a914-7e64c31972ae): Scheduling next merge selecting task after 60000ms | ||
| 6838 | 2026.08.18 00:05:54.454588 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 820 | ||
| 6839 | 2026.08.18 00:05:54.461823 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6840 | 2026.08.18 00:05:54.461837 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6841 | 2026.08.18 00:05:54.465245 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_109_109_0 (state Temporary) | ||
| 6842 | 2026.08.18 00:05:54.467835 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_109_109_0 to 202608_110_110_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6843 | 2026.08.18 00:05:54.468586 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 820 | ||
| 6844 | 2026.08.18 00:05:55.000297 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 573.70 MiB, peak 597.37 MiB, free memory in arenas 138.95 MiB, will set to 575.20 MiB (RSS), difference: 1.51 MiB | ||
| 6845 | 2026.08.18 00:06:00.000238 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 575.20 MiB, peak 597.37 MiB, free memory in arenas 137.94 MiB, will set to 574.16 MiB (RSS), difference: -1.05 MiB | ||
| 6846 | 2026.08.18 00:06:00.333654 [ 432039 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6847 | 2026.08.18 00:06:00.333681 [ 432039 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6848 | 2026.08.18 00:06:00.579110 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 262931 | ||
| 6849 | 2026.08.18 00:06:00.580825 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6850 | 2026.08.18 00:06:00.580846 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6851 | 2026.08.18 00:06:00.581705 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_110_110_0 (state Temporary) | ||
| 6852 | 2026.08.18 00:06:00.581920 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_110_110_0 to 202608_111_111_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6853 | 2026.08.18 00:06:00.582044 [ 432043 ] {} <Debug> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Selected 6 parts from 202608_1_106_21 to 202608_111_111_0 | ||
| 6854 | 2026.08.18 00:06:00.582046 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 262931 | ||
| 6855 | 2026.08.18 00:06:00.582068 [ 432043 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6856 | 2026.08.18 00:06:00.582078 [ 432043 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6857 | 2026.08.18 00:06:00.582184 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_106_21 to 202608_111_111_0 into Compact with storage Full | ||
| 6858 | 2026.08.18 00:06:00.582279 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6859 | 2026.08.18 00:06:00.582301 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 32 marks from part 202608_1_106_21, total 253377 rows starting from the beginning of the part | ||
| 6860 | 2026.08.18 00:06:00.582353 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_107_107_0, total 2254 rows starting from the beginning of the part | ||
| 6861 | 2026.08.18 00:06:00.582380 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_108_108_0, total 2576 rows starting from the beginning of the part | ||
| 6862 | 2026.08.18 00:06:00.582401 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_109_109_0, total 2254 rows starting from the beginning of the part | ||
| 6863 | 2026.08.18 00:06:00.582425 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_110_110_0, total 2576 rows starting from the beginning of the part | ||
| 6864 | 2026.08.18 00:06:00.582449 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_111_111_0, total 2254 rows starting from the beginning of the part | ||
| 6865 | 2026.08.18 00:06:00.602122 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 265291 rows, containing 4 columns (4 merged, 0 gathered) in 0.019987964 sec., 13272537.413015151 rows/sec., 190.43 MiB/sec. | ||
| 6866 | 2026.08.18 00:06:00.602288 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Trace> MergedBlockOutputStream: filled checksums 202608_1_111_22 (state Temporary) | ||
| 6867 | 2026.08.18 00:06:00.602417 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_merge_202608_1_111_22 to 202608_1_111_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6868 | 2026.08.18 00:06:00.602440 [ 432224 ] {ad9e363f-9d36-4a06-9162-b016a7de0753::202608_1_111_22} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753) (MergerMutator): Merged 6 parts: [202608_1_106_21, 202608_111_111_0] -> 202608_1_111_22 | ||
| 6869 | 2026.08.18 00:06:00.602485 [ 432224 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.76 MiB. | ||
| 6870 | 2026.08.18 00:06:01.968761 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 827 | ||
| 6871 | 2026.08.18 00:06:01.976046 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6872 | 2026.08.18 00:06:01.976060 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6873 | 2026.08.18 00:06:01.979497 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_110_110_0 (state Temporary) | ||
| 6874 | 2026.08.18 00:06:01.981975 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_110_110_0 to 202608_111_111_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6875 | 2026.08.18 00:06:01.982113 [ 432049 ] {} <Debug> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Selected 6 parts from 202608_1_106_21 to 202608_111_111_0 | ||
| 6876 | 2026.08.18 00:06:01.982137 [ 432049 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6877 | 2026.08.18 00:06:01.982143 [ 432049 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6878 | 2026.08.18 00:06:01.982211 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Debug> MergeTask::PrepareStage: Merging 6 parts: from 202608_1_106_21 to 202608_111_111_0 into Compact with storage Full | ||
| 6879 | 2026.08.18 00:06:01.982811 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 827 | ||
| 6880 | 2026.08.18 00:06:01.985602 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Debug> MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal | ||
| 6881 | 2026.08.18 00:06:01.985912 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_1_106_21, total 798 rows starting from the beginning of the part | ||
| 6882 | 2026.08.18 00:06:01.987650 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_107_107_0, total 7 rows starting from the beginning of the part | ||
| 6883 | 2026.08.18 00:06:01.989350 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_108_108_0, total 8 rows starting from the beginning of the part | ||
| 6884 | 2026.08.18 00:06:01.991112 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_109_109_0, total 7 rows starting from the beginning of the part | ||
| 6885 | 2026.08.18 00:06:01.993037 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_110_110_0, total 8 rows starting from the beginning of the part | ||
| 6886 | 2026.08.18 00:06:01.994743 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Debug> MergeTreeSequentialSource: Reading 2 marks from part 202608_111_111_0, total 7 rows starting from the beginning of the part | ||
| 6887 | 2026.08.18 00:06:02.000187 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 584.71 MiB, peak 597.37 MiB, free memory in arenas 122.83 MiB, will set to 571.18 MiB (RSS), difference: -13.54 MiB | ||
| 6888 | 2026.08.18 00:06:02.022310 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Debug> MergeTask::MergeProjectionsStage: Merge sorted 835 rows, containing 679 columns (679 merged, 0 gathered) in 0.040129669 sec., 20807.54765258592 rows/sec., 11.85 MiB/sec. | ||
| 6889 | 2026.08.18 00:06:02.023338 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Trace> MergedBlockOutputStream: filled checksums 202608_1_111_22 (state Temporary) | ||
| 6890 | 2026.08.18 00:06:02.026739 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_merge_202608_1_111_22 to 202608_1_111_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6891 | 2026.08.18 00:06:02.026779 [ 432216 ] {de328819-5068-4a62-b1b9-3decbc1f6c88::202608_1_111_22} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88) (MergerMutator): Merged 6 parts: [202608_1_106_21, 202608_111_111_0] -> 202608_1_111_22 | ||
| 6892 | 2026.08.18 00:06:02.027280 [ 432216 ] {} <Debug> MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.63 MiB. | ||
| 6893 | 2026.08.18 00:06:03.000183 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 561.30 MiB, peak 597.37 MiB, free memory in arenas 132.56 MiB, will set to 576.31 MiB (RSS), difference: 15.01 MiB | ||
| 6894 | 2026.08.18 00:06:08.082234 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2576 entries to flush up to offset 265507 | ||
| 6895 | 2026.08.18 00:06:08.083443 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6896 | 2026.08.18 00:06:08.083455 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6897 | 2026.08.18 00:06:08.083902 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_111_111_0 (state Temporary) | ||
| 6898 | 2026.08.18 00:06:08.084037 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_111_111_0 to 202608_112_112_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6899 | 2026.08.18 00:06:08.084126 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 265507 | ||
| 6900 | 2026.08.18 00:06:09.482988 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 835 | ||
| 6901 | 2026.08.18 00:06:09.490456 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6902 | 2026.08.18 00:06:09.490471 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6903 | 2026.08.18 00:06:09.493801 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_111_111_0 (state Temporary) | ||
| 6904 | 2026.08.18 00:06:09.496291 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_111_111_0 to 202608_112_112_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6905 | 2026.08.18 00:06:09.497062 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 835 | ||
| 6906 | 2026.08.18 00:06:15.000207 [ 432270 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 576.15 MiB, peak 597.37 MiB, free memory in arenas 131.59 MiB, will set to 568.29 MiB (RSS), difference: -7.86 MiB | ||
| 6907 | 2026.08.18 00:06:15.333781 [ 432155 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 6908 | 2026.08.18 00:06:15.333805 [ 432155 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 6909 | 2026.08.18 00:06:15.584306 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 2254 entries to flush up to offset 267761 | ||
| 6910 | 2026.08.18 00:06:15.585431 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6911 | 2026.08.18 00:06:15.585445 [ 432255 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6912 | 2026.08.18 00:06:15.585920 [ 432255 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_112_112_0 (state Temporary) | ||
| 6913 | 2026.08.18 00:06:15.586063 [ 432255 ] {} <Trace> system.asynchronous_metric_log (ad9e363f-9d36-4a06-9162-b016a7de0753): Renaming temporary part tmp_insert_202608_112_112_0 to 202608_113_113_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6914 | 2026.08.18 00:06:15.586154 [ 432255 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 267761 | ||
| 6915 | 2026.08.18 00:06:16.000669 [ 432270 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 7.709e-05 sec. | ||
| 6916 | 2026.08.18 00:06:16.997234 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 7 entries to flush up to offset 842 | ||
| 6917 | 2026.08.18 00:06:17.004142 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 6918 | 2026.08.18 00:06:17.004157 [ 432256 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 136.62 GiB. | ||
| 6919 | 2026.08.18 00:06:17.007497 [ 432256 ] {} <Trace> MergedBlockOutputStream: filled checksums 202608_112_112_0 (state Temporary) | ||
| 6920 | 2026.08.18 00:06:17.010067 [ 432256 ] {} <Trace> system.metric_log (de328819-5068-4a62-b1b9-3decbc1f6c88): Renaming temporary part tmp_insert_202608_112_112_0 to 202608_113_113_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 6921 | 2026.08.18 00:06:17.010857 [ 432256 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 842 | ||