| 1 | 2024.12.18 23:38:18.050568 [ 1 ] {} <Information> Application: Will watch for the process with pid 9279 | ||
| 2 | 2024.12.18 23:38:18.050829 [ 1 ] {} <Information> Application: Forked a child process to watch | ||
| 3 | 2024.12.18 23:38:18.205731 [ 1 ] {} <Information> Application: Starting ClickHouse 23.8.7.1 (revision: 54477, git hash: b6a391579f09dde47a1aa5602e154e7fcde80d1e, build id: <unknown>), PID 9279 | ||
| 4 | 2024.12.18 23:38:18.205885 [ 1 ] {} <Information> Application: starting up | ||
| 5 | 2024.12.18 23:38:18.206226 [ 1 ] {} <Information> Application: OS name: SunOS, version: 5.11, architecture: i86pc | ||
| 6 | 2024.12.18 23:38:18.374614 [ 1 ] {} <Information> Application: Available RAM: 31.32 GiB; physical cores: 8; logical cores: 8. | ||
| 7 | 2024.12.18 23:38:18.379213 [ 1 ] {} <Debug> Application: rlimit on number of file descriptors is 65536 | ||
| 8 | 2024.12.18 23:38:18.379228 [ 1 ] {} <Debug> Application: Initializing DateLUT. | ||
| 9 | 2024.12.18 23:38:18.379239 [ 1 ] {} <Trace> Application: Initialized DateLUT with time zone 'UTC'. | ||
| 10 | 2024.12.18 23:38:18.379285 [ 1 ] {} <Debug> Context: Setting up /var/tmp/omicron_tmp/clickward_test/clickhouse-1/data/tmp/ to store temporary data in it | ||
| 11 | 2024.12.18 23:38:18.379603 [ 1 ] {} <Debug> Application: Initializing interserver credentials. | ||
| 12 | 2024.12.18 23:38:18.381613 [ 1 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/clickward_test/clickhouse-1/clickhouse-config.xml' | ||
| 13 | 2024.12.18 23:38:18.382633 [ 1 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/clickward_test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 14 | 2024.12.18 23:38:18.383829 [ 1 ] {} <Information> Application: Setting max_server_memory_usage was set to 28.18 GiB (31.32 GiB available * 0.90 max_server_memory_usage_to_ram_ratio) | ||
| 15 | 2024.12.18 23:38:18.383844 [ 1 ] {} <Information> Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.66 GiB (31.32 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) | ||
| 16 | 2024.12.18 23:38:18.383863 [ 1 ] {} <Information> Application: Merges and mutations memory limit is set to 15.66 GiB | ||
| 17 | 2024.12.18 23:38:18.391174 [ 1 ] {} <Information> BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 18 | 2024.12.18 23:38:18.392530 [ 1 ] {} <Information> BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 128 threads | ||
| 19 | 2024.12.18 23:38:18.401396 [ 1 ] {} <Information> BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 20 | 2024.12.18 23:38:18.402969 [ 1 ] {} <Information> BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 21 | 2024.12.18 23:38:18.404941 [ 1 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. | ||
| 22 | 2024.12.18 23:38:18.405022 [ 1 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/clickward_test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 23 | 2024.12.18 23:38:18.407617 [ 1 ] {} <Information> Application: Listening for replica communication (interserver): http://[::1]:29401 | ||
| 24 | 2024.12.18 23:38:18.408523 [ 1 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/clickward_test/clickhouse-1/clickhouse-config.xml' | ||
| 25 | 2024.12.18 23:38:18.409284 [ 1 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/clickward_test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 26 | 2024.12.18 23:38:18.418952 [ 1 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/clickward_test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 27 | 2024.12.18 23:38:18.419550 [ 1 ] {} <Debug> Access(user directories): Added users_xml access storage 'users_xml', path: /var/tmp/omicron_tmp/clickward_test/clickhouse-1/clickhouse-config.xml | ||
| 28 | 2024.12.18 23:38:18.421093 [ 1 ] {} <Information> Context: Initialized background executor for merges and mutations with num_threads=16, num_tasks=32, scheduling_policy=round_robin | ||
| 29 | 2024.12.18 23:38:18.421904 [ 1 ] {} <Information> Context: Initialized background executor for move operations with num_threads=8, num_tasks=8 | ||
| 30 | 2024.12.18 23:38:18.422635 [ 1 ] {} <Information> Context: Initialized background executor for fetches with num_threads=8, num_tasks=8 | ||
| 31 | 2024.12.18 23:38:18.423393 [ 1 ] {} <Information> Context: Initialized background executor for common operations (e.g. clearing old parts) with num_threads=8, num_tasks=8 | ||
| 32 | 2024.12.18 23:38:18.424695 [ 1 ] {} <Information> DNSCacheUpdater: Update period 15 seconds | ||
| 33 | 2024.12.18 23:38:18.424726 [ 1 ] {} <Information> Application: Loading metadata from /var/tmp/omicron_tmp/clickward_test/clickhouse-1/data/ | ||
| 34 | 2024.12.18 23:38:18.424732 [ 178 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 35 | 2024.12.18 23:38:18.424775 [ 178 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 36 | 2024.12.18 23:38:18.433379 [ 1 ] {} <Information> DatabaseAtomic (system): Metadata processed, database system has 0 tables and 0 dictionaries in total. | ||
| 37 | 2024.12.18 23:38:18.433397 [ 1 ] {} <Information> TablesLoader: Parsed metadata of 0 tables in 1 databases in 9.9093e-05 sec | ||
| 38 | 2024.12.18 23:38:18.433418 [ 1 ] {} <Trace> ReferentialDeps: No tables | ||
| 39 | 2024.12.18 23:38:18.433428 [ 1 ] {} <Trace> LoadingDeps: No tables | ||
| 40 | 2024.12.18 23:38:18.434303 [ 1 ] {} <Information> DatabaseAtomic (system): Starting up tables. | ||
| 41 | 2024.12.18 23:38:18.436832 [ 1 ] {} <Debug> SystemLog: Not creating system.query_log since corresponding section 'query_log' is missing from config | ||
| 42 | 2024.12.18 23:38:18.436847 [ 1 ] {} <Debug> SystemLog: Not creating system.query_thread_log since corresponding section 'query_thread_log' is missing from config | ||
| 43 | 2024.12.18 23:38:18.436895 [ 1 ] {} <Debug> SystemLog: Not creating system.part_log since corresponding section 'part_log' is missing from config | ||
| 44 | 2024.12.18 23:38:18.436911 [ 1 ] {} <Debug> SystemLog: Not creating system.trace_log since corresponding section 'trace_log' is missing from config | ||
| 45 | 2024.12.18 23:38:18.436917 [ 1 ] {} <Debug> SystemLog: Not creating system.crash_log since corresponding section 'crash_log' is missing from config | ||
| 46 | 2024.12.18 23:38:18.436951 [ 1 ] {} <Debug> SystemLog: Not creating system.text_log since corresponding section 'text_log' is missing from config | ||
| 47 | 2024.12.18 23:38:18.436966 [ 1 ] {} <Debug> SystemLog: Creating system.metric_log from metric_log | ||
| 48 | 2024.12.18 23:38:18.452089 [ 1 ] {} <Debug> SystemLog: Not creating system.filesystem_cache_log since corresponding section 'filesystem_cache_log' is missing from config | ||
| 49 | 2024.12.18 23:38:18.452147 [ 1 ] {} <Debug> SystemLog: Not creating system.filesystem_read_prefetches_log since corresponding section 'filesystem_read_prefetches_log' is missing from config | ||
| 50 | 2024.12.18 23:38:18.452163 [ 1 ] {} <Debug> SystemLog: Creating system.asynchronous_metric_log from asynchronous_metric_log | ||
| 51 | 2024.12.18 23:38:18.452378 [ 1 ] {} <Debug> SystemLog: Creating system.opentelemetry_span_log from opentelemetry_span_log | ||
| 52 | 2024.12.18 23:38:18.456754 [ 1 ] {} <Debug> SystemLog: Not creating system.query_views_log since corresponding section 'query_views_log' is missing from config | ||
| 53 | 2024.12.18 23:38:18.456860 [ 1 ] {} <Debug> SystemLog: Not creating system.zookeeper_log since corresponding section 'zookeeper_log' is missing from config | ||
| 54 | 2024.12.18 23:38:18.456891 [ 1 ] {} <Debug> SystemLog: Not creating system.session_log since corresponding section 'session_log' is missing from config | ||
| 55 | 2024.12.18 23:38:18.456907 [ 1 ] {} <Debug> SystemLog: Not creating system.transactions_info_log since corresponding section 'transactions_info_log' is missing from config | ||
| 56 | 2024.12.18 23:38:18.456963 [ 1 ] {} <Debug> SystemLog: Not creating system.processors_profile_log since corresponding section 'processors_profile_log' is missing from config | ||
| 57 | 2024.12.18 23:38:18.456989 [ 1 ] {} <Debug> SystemLog: Not creating system.asynchronous_insert_log since corresponding section 'asynchronous_insert_log' is missing from config | ||
| 58 | 2024.12.18 23:38:18.496017 [ 1 ] {} <Information> DatabaseCatalog: Found 0 partially dropped tables. Will load them and retry removal. | ||
| 59 | 2024.12.18 23:38:18.496467 [ 1 ] {} <Information> DatabaseAtomic (default): Metadata processed, database default has 0 tables and 0 dictionaries in total. | ||
| 60 | 2024.12.18 23:38:18.496483 [ 1 ] {} <Information> TablesLoader: Parsed metadata of 0 tables in 1 databases in 3.6402e-05 sec | ||
| 61 | 2024.12.18 23:38:18.496490 [ 1 ] {} <Trace> ReferentialDeps: No tables | ||
| 62 | 2024.12.18 23:38:18.496495 [ 1 ] {} <Trace> LoadingDeps: No tables | ||
| 63 | 2024.12.18 23:38:18.496509 [ 1 ] {} <Information> DatabaseAtomic (default): Starting up tables. | ||
| 64 | 2024.12.18 23:38:18.496705 [ 1 ] {} <Information> UserDefinedSQLObjectsLoaderFromDisk: Loading user defined objects from /var/tmp/omicron_tmp/clickward_test/clickhouse-1/data/user_defined/ | ||
| 65 | 2024.12.18 23:38:18.496748 [ 1 ] {} <Debug> UserDefinedSQLObjectsLoaderFromDisk: User defined objects loaded | ||
| 66 | 2024.12.18 23:38:18.496755 [ 1 ] {} <Debug> Application: Loaded metadata. | ||
| 67 | 2024.12.18 23:38:18.496762 [ 1 ] {} <Information> Application: Query Profiler and TraceCollector are disabled because they require PHDR cache to be created (otherwise the function 'dl_iterate_phdr' is not lock free and not async-signal safe). | ||
| 68 | 2024.12.18 23:38:18.496767 [ 1 ] {} <Information> Application: TaskStats is not implemented for this OS. IO accounting will be disabled. | ||
| 69 | 2024.12.18 23:38:18.496870 [ 1 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. | ||
| 70 | 2024.12.18 23:38:18.497029 [ 1 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 13.98 MiB, peak 16.16 MiB, free memory in arenas 0.00 B, will set to 108.09 MiB (RSS), difference: 94.11 MiB | ||
| 71 | 2024.12.18 23:38:18.499154 [ 1 ] {} <Trace> AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 2.7897e-05 sec. | ||
| 72 | 2024.12.18 23:38:18.501340 [ 1 ] {} <Information> Application: Listening for http://[::1]:29301 | ||
| 73 | 2024.12.18 23:38:18.501358 [ 252 ] {} <Debug> DDLWorker: Starting DDLWorker thread | ||
| 74 | 2024.12.18 23:38:18.501384 [ 253 ] {} <Debug> DDLWorker: Started DDLWorker cleanup thread | ||
| 75 | 2024.12.18 23:38:18.501403 [ 252 ] {} <Debug> DDLWorker: Initializing DDLWorker thread | ||
| 76 | 2024.12.18 23:38:18.501490 [ 1 ] {} <Information> Application: Listening for native protocol (tcp): [::1]:29201 | ||
| 77 | 2024.12.18 23:38:18.547787 [ 1 ] {} <Information> Application: Ready for connections. | ||
| 78 | 2024.12.18 23:38:18.604265 [ 252 ] {} <Error> virtual bool DB::DDLWorker::initializeMainThread(): Code: 999. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: [::1]:29002, [::1]:29001, [::1]:29003 | ||
| 79 | Poco::Exception. Code: 1000, e.code() = 146, Connection refused: [::1]:29002 (version 23.8.7.1), [::1]:29002 | ||
| 80 | Poco::Exception. Code: 1000, e.code() = 146, Connection refused: [::1]:29001 (version 23.8.7.1), [::1]:29001 | ||
| 81 | Poco::Exception. Code: 1000, e.code() = 146, Connection refused: [::1]:29003 (version 23.8.7.1), [::1]:29003 | ||
| 82 | Poco::Exception. Code: 1000, e.code() = 146, Connection refused: [::1]:29002 (version 23.8.7.1), [::1]:29002 | ||
| 83 | Poco::Exception. Code: 1000, e.code() = 146, Connection refused: [::1]:29001 (version 23.8.7.1), [::1]:29001 | ||
| 84 | Poco::Exception. Code: 1000, e.code() = 146, Connection refused: [::1]:29003 (version 23.8.7.1), [::1]:29003 | ||
| 85 | Poco::Exception. Code: 1000, e.code() = 146, Connection refused: [::1]:29002 (version 23.8.7.1), [::1]:29002 | ||
| 86 | Poco::Exception. Code: 1000, e.code() = 146, Connection refused (version 23.8.7.1), [::1]:29001 | ||
| 87 | Poco::Exception. Code: 1000, e.code() = 146, Connection refused: [::1]:29003 (version 23.8.7.1), [::1]:29003 | ||
| 88 | . (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): | ||
| 89 | |||
| 90 | 0. StackTrace::tryCapture() @ 0x0000000010014f57 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 91 | 1. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000ffe797f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 92 | 2. DB::Exception::Exception<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(int, FormatStringHelperImpl<std::type_identity<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>::type>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) @ 0x000000000aded677 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 93 | 3. Coordination::Exception::Exception<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>::type>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) @ 0x0000000015562ded in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 94 | 4. Coordination::ZooKeeper::connect(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node> > const&, Poco::Timespan) @ 0x0000000016bad2ca in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 95 | 5. Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node> > const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)> >&&) @ 0x0000000016baaecb in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 96 | 6. zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs) @ 0x0000000016b741e9 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 97 | 7. zkutil::ZooKeeper::ZooKeeper(Poco::Util::AbstractConfiguration const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<DB::ZooKeeperLog>) @ 0x0000000016b75a63 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 98 | 8. DB::Context::getZooKeeper() const @ 0x0000000014e80509 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 99 | 9. DB::DDLWorker::getAndSetZooKeeper() @ 0x0000000014ebd50a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 100 | 10. DB::DDLWorker::initializeMainThread() @ 0x0000000014ecbd0c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 101 | 11. DB::DDLWorker::runMainThread() @ 0x0000000014ebbd51 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 102 | 12. ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::{lambda()#1}::operator()() @ 0x0000000014ecd995 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 103 | 13. ThreadPoolImpl<std::thread>::worker(std::_List_iterator<std::thread>) @ 0x0000000019092c5c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 104 | 14. ? @ 0x000000000014d750 in /usr/gcc/13/lib/amd64/libstdc++.so.6.0.32 | ||
| 105 | 15. _thrp_setup @ 0x0000000000114f37 in /lib/amd64/libc.so.1 | ||
| 106 | (version 23.8.7.1) | ||
| 107 | 2024.12.18 23:38:19.000234 [ 248 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 108.12 MiB, peak 108.12 MiB, free memory in arenas 0.00 B, will set to 117.21 MiB (RSS), difference: 9.09 MiB | ||
| 108 | 2024.12.18 23:38:20.144129 [ 3 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:40378 | ||
| 109 | 2024.12.18 23:38:20.144320 [ 3 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 110 | 2024.12.18 23:38:20.144394 [ 3 ] {} <Debug> TCP-Session: 6f47ffbf-61d6-488f-a01c-5594b9bc42a8 Authenticating user 'default' from [::1]:40378 | ||
| 111 | 2024.12.18 23:38:20.144461 [ 3 ] {} <Debug> TCP-Session: 6f47ffbf-61d6-488f-a01c-5594b9bc42a8 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 112 | 2024.12.18 23:38:20.144475 [ 3 ] {} <Debug> TCP-Session: 6f47ffbf-61d6-488f-a01c-5594b9bc42a8 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 113 | 2024.12.18 23:38:20.144664 [ 3 ] {} <Trace> ContextAccess (default): Settings: readonly = 0, allow_ddl = true, allow_introspection_functions = false | ||
| 114 | 2024.12.18 23:38:20.144707 [ 3 ] {} <Trace> ContextAccess (default): List of all grants: GRANT SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* | ||
| 115 | 2024.12.18 23:38:20.144726 [ 3 ] {} <Trace> ContextAccess (default): List of all grants including implicit: GRANT SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* | ||
| 116 | 2024.12.18 23:38:20.145375 [ 4 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:41434 | ||
| 117 | 2024.12.18 23:38:20.145417 [ 5 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:53819 | ||
| 118 | 2024.12.18 23:38:20.145583 [ 4 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 119 | 2024.12.18 23:38:20.145596 [ 5 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 120 | 2024.12.18 23:38:20.145628 [ 4 ] {} <Debug> TCP-Session: 09e6fc92-66f1-4c05-aef2-9d2e551984f5 Authenticating user 'default' from [::1]:41434 | ||
| 121 | 2024.12.18 23:38:20.145642 [ 5 ] {} <Debug> TCP-Session: e0109645-10e3-48e6-95fb-3011656ede56 Authenticating user 'default' from [::1]:53819 | ||
| 122 | 2024.12.18 23:38:20.145650 [ 4 ] {} <Debug> TCP-Session: 09e6fc92-66f1-4c05-aef2-9d2e551984f5 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 123 | 2024.12.18 23:38:20.145664 [ 5 ] {} <Debug> TCP-Session: e0109645-10e3-48e6-95fb-3011656ede56 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 124 | 2024.12.18 23:38:20.145670 [ 4 ] {} <Debug> TCP-Session: 09e6fc92-66f1-4c05-aef2-9d2e551984f5 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 125 | 2024.12.18 23:38:20.145679 [ 5 ] {} <Debug> TCP-Session: e0109645-10e3-48e6-95fb-3011656ede56 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 126 | 2024.12.18 23:38:20.145843 [ 256 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:49920 | ||
| 127 | 2024.12.18 23:38:20.145999 [ 256 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 128 | 2024.12.18 23:38:20.146026 [ 256 ] {} <Debug> TCP-Session: ea589be7-6cb4-4308-92cb-72258a07ccc3 Authenticating user 'default' from [::1]:49920 | ||
| 129 | 2024.12.18 23:38:20.146046 [ 256 ] {} <Debug> TCP-Session: ea589be7-6cb4-4308-92cb-72258a07ccc3 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 130 | 2024.12.18 23:38:20.146065 [ 256 ] {} <Debug> TCP-Session: ea589be7-6cb4-4308-92cb-72258a07ccc3 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 131 | 2024.12.18 23:38:20.146065 [ 258 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:59605 | ||
| 132 | 2024.12.18 23:38:20.146077 [ 257 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:61453 | ||
| 133 | 2024.12.18 23:38:20.146335 [ 259 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:46978 | ||
| 134 | 2024.12.18 23:38:20.146342 [ 257 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 135 | 2024.12.18 23:38:20.146348 [ 258 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 136 | 2024.12.18 23:38:20.146373 [ 257 ] {} <Debug> TCP-Session: 43d585db-31f4-4bd7-aa31-6d346372e938 Authenticating user 'default' from [::1]:61453 | ||
| 137 | 2024.12.18 23:38:20.146410 [ 258 ] {} <Debug> TCP-Session: f017b1be-55be-4de4-8777-39d5a4c5f660 Authenticating user 'default' from [::1]:59605 | ||
| 138 | 2024.12.18 23:38:20.146420 [ 257 ] {} <Debug> TCP-Session: 43d585db-31f4-4bd7-aa31-6d346372e938 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 139 | 2024.12.18 23:38:20.146479 [ 257 ] {} <Debug> TCP-Session: 43d585db-31f4-4bd7-aa31-6d346372e938 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 140 | 2024.12.18 23:38:20.146479 [ 258 ] {} <Debug> TCP-Session: f017b1be-55be-4de4-8777-39d5a4c5f660 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 141 | 2024.12.18 23:38:20.146489 [ 259 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 142 | 2024.12.18 23:38:20.146511 [ 260 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:40600 | ||
| 143 | 2024.12.18 23:38:20.146527 [ 259 ] {} <Debug> TCP-Session: 18569ebb-e8bd-492a-b770-5f468b5df064 Authenticating user 'default' from [::1]:46978 | ||
| 144 | 2024.12.18 23:38:20.146522 [ 258 ] {} <Debug> TCP-Session: f017b1be-55be-4de4-8777-39d5a4c5f660 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 145 | 2024.12.18 23:38:20.146585 [ 259 ] {} <Debug> TCP-Session: 18569ebb-e8bd-492a-b770-5f468b5df064 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 146 | 2024.12.18 23:38:20.146649 [ 259 ] {} <Debug> TCP-Session: 18569ebb-e8bd-492a-b770-5f468b5df064 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 147 | 2024.12.18 23:38:20.146659 [ 261 ] {} <Trace> TCPHandlerFactory: TCP Request. Address: [::1]:34729 | ||
| 148 | 2024.12.18 23:38:20.146738 [ 260 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 149 | 2024.12.18 23:38:20.146778 [ 260 ] {} <Debug> TCP-Session: c46ce648-59ca-4fd1-b2d9-734886b8ddff Authenticating user 'default' from [::1]:40600 | ||
| 150 | 2024.12.18 23:38:20.148975 [ 261 ] {} <Debug> TCPHandler: Connected oximeter version 23.8.0, revision: 54465, database: default, user: default. | ||
| 151 | 2024.12.18 23:38:20.148982 [ 260 ] {} <Debug> TCP-Session: c46ce648-59ca-4fd1-b2d9-734886b8ddff Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 152 | 2024.12.18 23:38:20.149011 [ 261 ] {} <Debug> TCP-Session: 2ebbdf7b-363f-4430-87f8-d626ef02780d Authenticating user 'default' from [::1]:34729 | ||
| 153 | 2024.12.18 23:38:20.149020 [ 260 ] {} <Debug> TCP-Session: c46ce648-59ca-4fd1-b2d9-734886b8ddff Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 154 | 2024.12.18 23:38:20.149027 [ 261 ] {} <Debug> TCP-Session: 2ebbdf7b-363f-4430-87f8-d626ef02780d Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db | ||
| 155 | 2024.12.18 23:38:20.151087 [ 261 ] {} <Debug> TCP-Session: 2ebbdf7b-363f-4430-87f8-d626ef02780d Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 156 | 2024.12.18 23:38:20.168418 [ 259 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::1]:46978). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): | ||
| 157 | |||
| 158 | 0. StackTrace::tryCapture() @ 0x0000000010014f57 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 159 | 1. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000ffe797f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 160 | 2. DB::NetException::NetException<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(int, FormatStringHelperImpl<std::type_identity<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::type, std::type_identity<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::type>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&) @ 0x0000000015b19afe in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 161 | 3. DB::ReadBufferFromPocoSocket::nextImpl() @ 0x00000000190bd7af in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 162 | 4. DB::TCPHandler::runImpl() @ 0x0000000015b25abc in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 163 | 5. DB::TCPHandler::run() @ 0x0000000015b3661c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 164 | 6. Poco::Net::TCPServerConnection::start() @ 0x000000001954534c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 165 | 7. Poco::Net::TCPServerDispatcher::run() @ 0x0000000019545821 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 166 | 8. Poco::PooledThread::run() @ 0x00000000195aa732 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 167 | 9. Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000195a81e5 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 168 | 10. _thrp_setup @ 0x0000000000114f37 in /lib/amd64/libc.so.1 | ||
| 169 | (version 23.8.7.1) | ||
| 170 | 2024.12.18 23:38:20.168481 [ 259 ] {} <Debug> TCP-Session: 18569ebb-e8bd-492a-b770-5f468b5df064 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 171 | 2024.12.18 23:38:20.338660 [ 3 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 172 | 2024.12.18 23:38:20.338698 [ 3 ] {} <Debug> TCP-Session: 6f47ffbf-61d6-488f-a01c-5594b9bc42a8 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 173 | 2024.12.18 23:38:20.338985 [ 4 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 174 | 2024.12.18 23:38:20.339016 [ 4 ] {} <Debug> TCP-Session: 09e6fc92-66f1-4c05-aef2-9d2e551984f5 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 175 | 2024.12.18 23:38:20.339136 [ 257 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 176 | 2024.12.18 23:38:20.339156 [ 257 ] {} <Debug> TCP-Session: 43d585db-31f4-4bd7-aa31-6d346372e938 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 177 | 2024.12.18 23:38:20.339232 [ 261 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 178 | 2024.12.18 23:38:20.339251 [ 261 ] {} <Debug> TCP-Session: 2ebbdf7b-363f-4430-87f8-d626ef02780d Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 179 | 2024.12.18 23:38:20.339318 [ 256 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 180 | 2024.12.18 23:38:20.339341 [ 256 ] {} <Debug> TCP-Session: ea589be7-6cb4-4308-92cb-72258a07ccc3 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 181 | 2024.12.18 23:38:20.339390 [ 258 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 182 | 2024.12.18 23:38:20.339413 [ 258 ] {} <Debug> TCP-Session: f017b1be-55be-4de4-8777-39d5a4c5f660 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 183 | 2024.12.18 23:38:20.339448 [ 5 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 184 | 2024.12.18 23:38:20.339474 [ 5 ] {} <Debug> TCP-Session: e0109645-10e3-48e6-95fb-3011656ede56 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 185 | 2024.12.18 23:38:20.339494 [ 260 ] {} <Debug> TCPHandler: Done processing connection. | ||
| 186 | 2024.12.18 23:38:20.339516 [ 260 ] {} <Debug> TCP-Session: c46ce648-59ca-4fd1-b2d9-734886b8ddff Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db | ||
| 187 | 2024.12.18 23:38:21.000229 [ 248 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 117.25 MiB, peak 119.29 MiB, free memory in arenas 0.00 B, will set to 118.45 MiB (RSS), difference: 1.19 MiB | ||
| 188 | 2024.12.18 23:38:23.606367 [ 252 ] {} <Information> ZooKeeperClient: Connected to ZooKeeper at [::1]:29001 with session_id 5 | ||
| 189 | 2024.12.18 23:38:23.606925 [ 252 ] {} <Information> ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled | ||
| 190 | 2024.12.18 23:38:23.606948 [ 252 ] {} <Information> ZooKeeperClient: Keeper feature flag MULTI_READ: enabled | ||
| 191 | 2024.12.18 23:38:23.606954 [ 252 ] {} <Information> ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: disabled | ||
| 192 | 2024.12.18 23:38:23.606994 [ 252 ] {} <Trace> ZooKeeper: Initialized, hosts: [::1]:29001,[::1]:29002,[::1]:29003 | ||
| 193 | 2024.12.18 23:38:23.614656 [ 252 ] {} <Debug> DDLWorker: Initialized DDLWorker thread | ||
| 194 | 2024.12.18 23:38:23.614702 [ 252 ] {} <Debug> DDLWorker: Scheduling tasks | ||
| 195 | 2024.12.18 23:38:23.614710 [ 252 ] {} <Trace> DDLWorker: Don't have unfinished tasks after restarting | ||
| 196 | 2024.12.18 23:38:23.614721 [ 253 ] {} <Debug> DDLWorker: Cleaning queue | ||
| 197 | 2024.12.18 23:38:23.614956 [ 252 ] {} <Trace> DDLWorker: scheduleTasks: initialized=true, size_before_filtering=0, queue_size=0, entries=none..none, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=none | ||
| 198 | 2024.12.18 23:38:23.614969 [ 252 ] {} <Debug> DDLWorker: No tasks to schedule | ||
| 199 | 2024.12.18 23:38:23.614975 [ 252 ] {} <Debug> DDLWorker: Waiting for queue updates | ||
| 200 | 2024.12.18 23:38:25.957449 [ 245 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 496 entries to flush up to offset 496 | ||
| 201 | 2024.12.18 23:38:25.957565 [ 244 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 8 | ||
| 202 | 2024.12.18 23:38:25.957740 [ 245 ] {} <Debug> SystemLog (system.asynchronous_metric_log): Creating new table system.asynchronous_metric_log for AsynchronousMetricLog | ||
| 203 | 2024.12.18 23:38:25.957749 [ 244 ] {} <Debug> SystemLog (system.metric_log): Creating new table system.metric_log for MetricLog | ||
| 204 | 2024.12.18 23:38:25.978231 [ 245 ] {} <Trace> StoragePolicy (default): Storage policy default created, total volumes 1 | ||
| 205 | 2024.12.18 23:38:25.978572 [ 245 ] {} <Debug> system.asynchronous_metric_log (c79a6df8-b664-413c-bcc4-f856746c9e94): Loading data parts | ||
| 206 | 2024.12.18 23:38:25.978585 [ 244 ] {} <Debug> system.metric_log (d6473194-18b1-4da9-8e9e-9bfedfaec74a): Loading data parts | ||
| 207 | 2024.12.18 23:38:25.979105 [ 244 ] {} <Debug> system.metric_log (d6473194-18b1-4da9-8e9e-9bfedfaec74a): There are no data parts | ||
| 208 | 2024.12.18 23:38:25.979106 [ 245 ] {} <Debug> system.asynchronous_metric_log (c79a6df8-b664-413c-bcc4-f856746c9e94): There are no data parts | ||
| 209 | 2024.12.18 23:38:25.989431 [ 245 ] {} <Trace> system.asynchronous_metric_log (c79a6df8-b664-413c-bcc4-f856746c9e94): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 210 | 2024.12.18 23:38:25.989481 [ 245 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 138.68 GiB. | ||
| 211 | 2024.12.18 23:38:25.991742 [ 245 ] {} <Trace> MergedBlockOutputStream: filled checksums 202412_1_1_0 (state Temporary) | ||
| 212 | 2024.12.18 23:38:25.994723 [ 245 ] {} <Trace> system.asynchronous_metric_log (c79a6df8-b664-413c-bcc4-f856746c9e94): Renaming temporary part tmp_insert_202412_1_1_0 to 202412_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 213 | 2024.12.18 23:38:25.995311 [ 245 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 496 | ||
| 214 | 2024.12.18 23:38:26.000216 [ 248 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 123.15 MiB, peak 131.32 MiB, free memory in arenas 0.00 B, will set to 128.97 MiB (RSS), difference: 5.82 MiB | ||
| 215 | 2024.12.18 23:38:26.005793 [ 244 ] {} <Trace> system.metric_log (d6473194-18b1-4da9-8e9e-9bfedfaec74a): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 216 | 2024.12.18 23:38:26.005826 [ 244 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 138.68 GiB. | ||
| 217 | 2024.12.18 23:38:26.016582 [ 244 ] {} <Trace> MergedBlockOutputStream: filled checksums 202412_1_1_0 (state Temporary) | ||
| 218 | 2024.12.18 23:38:26.029476 [ 244 ] {} <Trace> system.metric_log (d6473194-18b1-4da9-8e9e-9bfedfaec74a): Renaming temporary part tmp_insert_202412_1_1_0 to 202412_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 219 | 2024.12.18 23:38:26.032249 [ 244 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 8 | ||
| 220 | 2024.12.18 23:38:27.000227 [ 248 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 129.61 MiB, peak 137.97 MiB, free memory in arenas 0.00 B, will set to 134.52 MiB (RSS), difference: 4.91 MiB | ||
| 221 | 2024.12.18 23:38:33.424886 [ 153 ] {} <Debug> DNSResolver: Updating DNS cache | ||
| 222 | 2024.12.18 23:38:33.424957 [ 153 ] {} <Debug> DNSResolver: Updated DNS cache | ||
| 223 | 2024.12.18 23:38:33.495463 [ 245 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushing system log, 496 entries to flush up to offset 992 | ||
| 224 | 2024.12.18 23:38:33.496714 [ 245 ] {} <Trace> system.asynchronous_metric_log (c79a6df8-b664-413c-bcc4-f856746c9e94): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 225 | 2024.12.18 23:38:33.496743 [ 245 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 138.68 GiB. | ||
| 226 | 2024.12.18 23:38:33.497328 [ 245 ] {} <Trace> MergedBlockOutputStream: filled checksums 202412_2_2_0 (state Temporary) | ||
| 227 | 2024.12.18 23:38:33.497791 [ 245 ] {} <Trace> system.asynchronous_metric_log (c79a6df8-b664-413c-bcc4-f856746c9e94): Renaming temporary part tmp_insert_202412_2_2_0 to 202412_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 228 | 2024.12.18 23:38:33.498036 [ 245 ] {} <Trace> SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 992 | ||
| 229 | 2024.12.18 23:38:33.532334 [ 244 ] {} <Trace> SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 16 | ||
| 230 | 2024.12.18 23:38:33.553475 [ 244 ] {} <Trace> system.metric_log (d6473194-18b1-4da9-8e9e-9bfedfaec74a): Trying to reserve 1.00 MiB using storage policy from min volume index 0 | ||
| 231 | 2024.12.18 23:38:33.553511 [ 244 ] {} <Trace> DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 138.68 GiB. | ||
| 232 | 2024.12.18 23:38:33.562368 [ 244 ] {} <Trace> MergedBlockOutputStream: filled checksums 202412_2_2_0 (state Temporary) | ||
| 233 | 2024.12.18 23:38:33.572818 [ 244 ] {} <Trace> system.metric_log (d6473194-18b1-4da9-8e9e-9bfedfaec74a): Renaming temporary part tmp_insert_202412_2_2_0 to 202412_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). | ||
| 234 | 2024.12.18 23:38:33.575776 [ 244 ] {} <Trace> SystemLog (system.metric_log): Flushed system log up to offset 16 | ||
| 235 | 2024.12.18 23:38:34.000210 [ 248 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 134.36 MiB, peak 143.14 MiB, free memory in arenas 0.00 B, will set to 135.91 MiB (RSS), difference: 1.55 MiB | ||