Max level shown:
1 2026.06.22 00:29:39.786600 [ 397662 ] {} <Information> SentryWriter: Sending crash reports is disabled
2 2026.06.22 00:29:39.834251 [ 397662 ] {} <Information> Application: Starting ClickHouse 23.8.7.24 (revision: 54477, git hash: 812b95e14ba8cf744bf1d70e6de607cf130a79fa, build id: 6484B75ECB16F4FF87A88E10249A45E1BDBDE2E5), PID 397662
3 2026.06.22 00:29:39.834305 [ 397662 ] {} <Information> Application: starting up
4 2026.06.22 00:29:39.834317 [ 397662 ] {} <Information> Application: OS name: Linux, version: 5.15.0-1015-aws, architecture: x86_64
5 2026.06.22 00:29:39.836118 [ 397662 ] {} <Information> Application: Available RAM: 30.74 GiB; physical cores: 16; logical cores: 16.
6 2026.06.22 00:29:39.957872 [ 397662 ] {} <Information> Application: Integrity check of the executable successfully passed (checksum: 63E397DB90569A93F13765E06F4A1FBF)
7 2026.06.22 00:29:39.957924 [ 397662 ] {} <Information> Application: It looks like the process has no CAP_IPC_LOCK capability, binary mlock will be disabled. It could happen due to incorrect ClickHouse package installation. You could resolve the problem manually with 'sudo setcap cap_ipc_lock=+ep /work/oxidecomputer/omicron/out/clickhouse/clickhouse'. Note that it will not work on 'nosuid' mounted filesystems.
8 2026.06.22 00:29:39.966691 [ 397662 ] {} <Information> SensitiveDataMaskerConfigRead: 1 query masking rules loaded.
9 2026.06.22 00:29:39.967440 [ 397662 ] {} <Information> Application: Setting max_server_memory_usage was set to 27.67 GiB (30.74 GiB available * 0.90 max_server_memory_usage_to_ram_ratio)
10 2026.06.22 00:29:39.967452 [ 397662 ] {} <Information> Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.37 GiB (30.74 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio)
11 2026.06.22 00:29:39.967455 [ 397662 ] {} <Information> Application: Merges and mutations memory limit is set to 15.37 GiB
12 2026.06.22 00:29:39.967487 [ 397662 ] {} <Information> BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads
13 2026.06.22 00:29:39.967855 [ 397662 ] {} <Information> BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 128 threads
14 2026.06.22 00:29:40.009028 [ 397662 ] {} <Information> BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads
15 2026.06.22 00:29:40.010710 [ 397662 ] {} <Information> BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads
16 2026.06.22 00:29:40.011004 [ 397662 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again.
17 2026.06.22 00:29:40.027967 [ 397662 ] {} <Information> Context: Initialized background executor for merges and mutations with num_threads=16, num_tasks=32, scheduling_policy=round_robin
18 2026.06.22 00:29:40.028114 [ 397662 ] {} <Information> Context: Initialized background executor for move operations with num_threads=8, num_tasks=8
19 2026.06.22 00:29:40.068585 [ 397662 ] {} <Information> Context: Initialized background executor for fetches with num_threads=8, num_tasks=8
20 2026.06.22 00:29:40.085145 [ 397662 ] {} <Information> Context: Initialized background executor for common operations (e.g. clearing old parts) with num_threads=8, num_tasks=8
21 2026.06.22 00:29:40.085516 [ 397662 ] {} <Information> DNSCacheUpdater: Update period 15 seconds
22 2026.06.22 00:29:40.085534 [ 397662 ] {} <Information> Application: Loading metadata from /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.1-clickhouse-Q4iMVM/datastore/
23 2026.06.22 00:29:40.101398 [ 397662 ] {} <Information> DatabaseAtomic (system): Metadata processed, database system has 0 tables and 0 dictionaries in total.
24 2026.06.22 00:29:40.101422 [ 397662 ] {} <Information> TablesLoader: Parsed metadata of 0 tables in 1 databases in 7.053e-05 sec
25 2026.06.22 00:29:40.101441 [ 397662 ] {} <Information> DatabaseAtomic (system): Starting up tables.
26 2026.06.22 00:29:40.114603 [ 397662 ] {} <Information> DatabaseCatalog: Found 0 partially dropped tables. Will load them and retry removal.
27 2026.06.22 00:29:40.120379 [ 397662 ] {} <Information> DatabaseAtomic (default): Metadata processed, database default has 0 tables and 0 dictionaries in total.
28 2026.06.22 00:29:40.120401 [ 397662 ] {} <Information> TablesLoader: Parsed metadata of 0 tables in 1 databases in 5.183e-05 sec
29 2026.06.22 00:29:40.120418 [ 397662 ] {} <Information> DatabaseAtomic (default): Starting up tables.
30 2026.06.22 00:29:40.120479 [ 397662 ] {} <Information> UserDefinedSQLObjectsLoaderFromDisk: Loading user defined objects from /var/tmp/omicron_tmp/test_all_output-061737baf9f4dcc1-test_omdb_success_cases.397230.1-clickhouse-Q4iMVM/datastore/user_defined/
31 2026.06.22 00:29:40.120508 [ 397662 ] {} <Information> Application: Tasks stats provider: procfs
32 2026.06.22 00:29:40.120514 [ 397662 ] {} <Information> Application: It looks like the process has no CAP_SYS_NICE capability, the setting 'os_thread_priority' will have no effect. It could happen due to incorrect ClickHouse package installation. You could resolve the problem manually with 'sudo setcap cap_sys_nice=+ep /work/oxidecomputer/omicron/out/clickhouse/clickhouse'. Note that it will not work on 'nosuid' mounted filesystems.
33 2026.06.22 00:29:40.120719 [ 397662 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again.
34 2026.06.22 00:29:40.121791 [ 397662 ] {} <Information> Application: Listening for http://[::1]:45617
35 2026.06.22 00:29:40.121816 [ 397662 ] {} <Information> Application: Listening for native protocol (tcp): [::1]:42693
36 2026.06.22 00:29:40.121832 [ 397662 ] {} <Information> Application: Listening for http://127.0.0.1:43577
37 2026.06.22 00:29:40.121848 [ 397662 ] {} <Information> Application: Listening for native protocol (tcp): 127.0.0.1:34375
38 2026.06.22 00:29:40.121850 [ 397662 ] {} <Information> Application: Ready for connections.
39 2026.06.22 00:29:43.991916 [ 399590 ] {99a2239e-ff65-4b45-a91a-5f6bc12a761a} <Error> executeQuery: Code: 81. DB::Exception: Database oximeter does not exist. (UNKNOWN_DATABASE) (version 23.8.7.24 (official build)) (from [::1]:40156) (in query: SELECT MAX(value) AS max_version FROM oximeter.version;), Stack trace (when copying this message, always include the lines below):
40 
41 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
42 1. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x000000000715892d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
43 2. DB::DatabaseCatalog::getDatabase(String const&) const @ 0x000000001170be95 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
44 3. DB::Context::resolveStorageID(DB::StorageID, DB::Context::StorageNamespace) const @ 0x00000000116810de in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
45 4. DB::JoinedTables::getLeftTableStorage() @ 0x0000000011fde326 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
46 5. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x0000000011ee92e3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
47 6. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000011f9c308 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
48 7. DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x0000000011ea331e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
49 8. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122e55ea in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
50 9. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122e15d5 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
51 10. DB::TCPHandler::runImpl() @ 0x0000000013156f39 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
52 11. DB::TCPHandler::run() @ 0x00000000131691b9 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
53 12. Poco::Net::TCPServerConnection::start() @ 0x0000000015b5f7f4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
54 13. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b609f1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
55 14. Poco::PooledThread::run() @ 0x0000000015c97227 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
56 15. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c954fc in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
57 16. ? @ 0x00007f3aa2df1ac3 in ?
58 17. ? @ 0x00007f3aa2e838d0 in ?
59 
60 2026.06.22 00:29:43.992005 [ 399590 ] {99a2239e-ff65-4b45-a91a-5f6bc12a761a} <Error> TCPHandler: Code: 81. DB::Exception: Database oximeter does not exist. (UNKNOWN_DATABASE), Stack trace (when copying this message, always include the lines below):
61 
62 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
63 1. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x000000000715892d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
64 2. DB::DatabaseCatalog::getDatabase(String const&) const @ 0x000000001170be95 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
65 3. DB::Context::resolveStorageID(DB::StorageID, DB::Context::StorageNamespace) const @ 0x00000000116810de in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
66 4. DB::JoinedTables::getLeftTableStorage() @ 0x0000000011fde326 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
67 5. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x0000000011ee92e3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
68 6. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000011f9c308 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
69 7. DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x0000000011ea331e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
70 8. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122e55ea in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
71 9. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122e15d5 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
72 10. DB::TCPHandler::runImpl() @ 0x0000000013156f39 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
73 11. DB::TCPHandler::run() @ 0x00000000131691b9 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
74 12. Poco::Net::TCPServerConnection::start() @ 0x0000000015b5f7f4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
75 13. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b609f1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
76 14. Poco::PooledThread::run() @ 0x0000000015c97227 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
77 15. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c954fc in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
78 16. ? @ 0x00007f3aa2df1ac3 in ?
79 17. ? @ 0x00007f3aa2e838d0 in ?
80 
81 2026.06.22 00:29:43.994355 [ 399591 ] {c569247f-e07c-4dfa-bf8b-dab08444ff08} <Error> executeQuery: Code: 81. DB::Exception: Database oximeter does not exist. (UNKNOWN_DATABASE) (version 23.8.7.24 (official build)) (from [::1]:40158) (in query: SELECT MAX(value) AS max_version FROM oximeter.version;), Stack trace (when copying this message, always include the lines below):
82 
83 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
84 1. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x000000000715892d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
85 2. DB::DatabaseCatalog::getDatabase(String const&) const @ 0x000000001170be95 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
86 3. DB::Context::resolveStorageID(DB::StorageID, DB::Context::StorageNamespace) const @ 0x00000000116810de in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
87 4. DB::JoinedTables::getLeftTableStorage() @ 0x0000000011fde326 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
88 5. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x0000000011ee92e3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
89 6. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000011f9c308 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
90 7. DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x0000000011ea331e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
91 8. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122e55ea in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
92 9. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122e15d5 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
93 10. DB::TCPHandler::runImpl() @ 0x0000000013156f39 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
94 11. DB::TCPHandler::run() @ 0x00000000131691b9 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
95 12. Poco::Net::TCPServerConnection::start() @ 0x0000000015b5f7f4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
96 13. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b609f1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
97 14. Poco::PooledThread::run() @ 0x0000000015c97227 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
98 15. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c954fc in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
99 16. ? @ 0x00007f3aa2df1ac3 in ?
100 17. ? @ 0x00007f3aa2e838d0 in ?
101 
102 2026.06.22 00:29:43.995135 [ 399591 ] {c569247f-e07c-4dfa-bf8b-dab08444ff08} <Error> TCPHandler: Code: 81. DB::Exception: Database oximeter does not exist. (UNKNOWN_DATABASE), Stack trace (when copying this message, always include the lines below):
103 
104 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
105 1. DB::Exception::Exception<String>(int, FormatStringHelperImpl<std::type_identity<String>::type>, String&&) @ 0x000000000715892d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
106 2. DB::DatabaseCatalog::getDatabase(String const&) const @ 0x000000001170be95 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
107 3. DB::Context::resolveStorageID(DB::StorageID, DB::Context::StorageNamespace) const @ 0x00000000116810de in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
108 4. DB::JoinedTables::getLeftTableStorage() @ 0x0000000011fde326 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
109 5. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context> const&, std::optional<DB::Pipe>, std::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::shared_ptr<DB::PreparedSets>) @ 0x0000000011ee92e3 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
110 6. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::shared_ptr<DB::IAST> const&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::vector<String, std::allocator<String>> const&) @ 0x0000000011f9c308 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
111 7. DB::InterpreterFactory::get(std::shared_ptr<DB::IAST>&, std::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x0000000011ea331e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
112 8. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122e55ea in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
113 9. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122e15d5 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
114 10. DB::TCPHandler::runImpl() @ 0x0000000013156f39 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
115 11. DB::TCPHandler::run() @ 0x00000000131691b9 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
116 12. Poco::Net::TCPServerConnection::start() @ 0x0000000015b5f7f4 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
117 13. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b609f1 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
118 14. Poco::PooledThread::run() @ 0x0000000015c97227 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
119 15. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c954fc in /work/oxidecomputer/omicron/out/clickhouse/clickhouse
120 16. ? @ 0x00007f3aa2df1ac3 in ?
121 17. ? @ 0x00007f3aa2e838d0 in ?
122 
123 2026.06.22 00:29:44.002072 [ 399590 ] {ec318d94-4dde-4017-8c9b-f61cbf467a0a} <Information> DatabaseAtomic (oximeter): Metadata processed, database oximeter has 0 tables and 0 dictionaries in total.
124 2026.06.22 00:29:44.002096 [ 399590 ] {ec318d94-4dde-4017-8c9b-f61cbf467a0a} <Information> TablesLoader: Parsed metadata of 0 tables in 1 databases in 8.661e-05 sec
125 2026.06.22 00:29:44.002121 [ 399590 ] {ec318d94-4dde-4017-8c9b-f61cbf467a0a} <Information> DatabaseAtomic (oximeter): Starting up tables.