| 1 | 2026.08.17 23:52:47.706076 [ 433557 ] {} <Information> Application: Will watch for the process with pid 433567 | ||
| 2 | 2026.08.17 23:52:47.706122 [ 433567 ] {} <Information> Application: Forked a child process to watch | ||
| 3 | 2026.08.17 23:52:47.706214 [ 433567 ] {} <Information> SentryWriter: Sending crash reports is disabled | ||
| 4 | 2026.08.17 23:52:47.706268 [ 433567 ] {} <Trace> Pipe: Pipe capacity is 1.00 MiB | ||
| 5 | 2026.08.17 23:52:47.740237 [ 433567 ] {} <Information> Application: Starting ClickHouse 23.8.7.24 (revision: 54477, git hash: 812b95e14ba8cf744bf1d70e6de607cf130a79fa, build id: 6484B75ECB16F4FF87A88E10249A45E1BDBDE2E5), PID 433567 | ||
| 6 | 2026.08.17 23:52:47.741543 [ 433567 ] {} <Information> Application: starting up | ||
| 7 | 2026.08.17 23:52:47.741559 [ 433567 ] {} <Information> Application: OS name: Linux, version: 5.15.0-1015-aws, architecture: x86_64 | ||
| 8 | 2026.08.17 23:52:47.744277 [ 433567 ] {} <Information> Application: Available RAM: 30.74 GiB; physical cores: 16; logical cores: 16. | ||
| 9 | 2026.08.17 23:52:47.763174 [ 433567 ] {} <Trace> AsynchronousMetrics: Scanning /sys/class/thermal | ||
| 10 | 2026.08.17 23:52:47.763183 [ 433567 ] {} <Trace> AsynchronousMetrics: Scanning /sys/block | ||
| 11 | 2026.08.17 23:52:47.763209 [ 433567 ] {} <Trace> AsynchronousMetrics: Scanning /sys/devices/system/edac | ||
| 12 | 2026.08.17 23:52:47.763215 [ 433567 ] {} <Trace> AsynchronousMetrics: Scanning /sys/class/hwmon | ||
| 13 | 2026.08.17 23:52:47.867212 [ 433567 ] {} <Information> Application: Integrity check of the executable successfully passed (checksum: 63E397DB90569A93F13765E06F4A1FBF) | ||
| 14 | 2026.08.17 23:52:47.873660 [ 433567 ] {} <Debug> Application: Set max number of file descriptors to 524288 (was 1024). | ||
| 15 | 2026.08.17 23:52:47.873679 [ 433567 ] {} <Debug> Application: rlimit on number of threads is 125877 | ||
| 16 | 2026.08.17 23:52:47.873684 [ 433567 ] {} <Debug> Application: Initializing DateLUT. | ||
| 17 | 2026.08.17 23:52:47.873687 [ 433567 ] {} <Trace> Application: Initialized DateLUT with time zone 'UTC'. | ||
| 18 | 2026.08.17 23:52:47.873707 [ 433567 ] {} <Debug> Context: Setting up /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.433551.1-oximeter-clickward-test/clickhouse-1/data/tmp/ to store temporary data in it | ||
| 19 | 2026.08.17 23:52:47.873848 [ 433567 ] {} <Debug> Application: Initializing interserver credentials. | ||
| 20 | 2026.08.17 23:52:47.874028 [ 433567 ] {} <Debug> ConfigReloader: Loading config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.433551.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml' | ||
| 21 | 2026.08.17 23:52:47.874402 [ 433567 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.433551.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performing update on configuration | ||
| 22 | 2026.08.17 23:52:47.874693 [ 433567 ] {} <Information> Application: Setting max_server_memory_usage was set to 27.67 GiB (30.74 GiB available * 0.90 max_server_memory_usage_to_ram_ratio) | ||
| 23 | 2026.08.17 23:52:47.874697 [ 433567 ] {} <Information> Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.37 GiB (30.74 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) | ||
| 24 | 2026.08.17 23:52:47.874699 [ 433567 ] {} <Information> Application: Merges and mutations memory limit is set to 15.37 GiB | ||
| 25 | 2026.08.17 23:52:47.874869 [ 433567 ] {} <Information> BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 26 | 2026.08.17 23:52:47.875202 [ 433567 ] {} <Information> BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 128 threads | ||
| 27 | 2026.08.17 23:52:47.878039 [ 433567 ] {} <Information> BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 28 | 2026.08.17 23:52:47.878436 [ 433567 ] {} <Information> BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads | ||
| 29 | 2026.08.17 23:52:47.878862 [ 433567 ] {} <Information> CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. | ||
| 30 | 2026.08.17 23:52:47.878869 [ 433567 ] {} <Debug> ConfigReloader: Loaded config '/var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.433551.1-oximeter-clickward-test/clickhouse-1/clickhouse-config.xml', performed update on configuration | ||
| 31 | 2026.08.17 23:52:47.884577 [ 433567 ] {} <Trace> Context: Shutting down named sessions | ||
| 32 | 2026.08.17 23:52:47.884591 [ 433567 ] {} <Trace> Context: Shutting down database catalog | ||
| 33 | 2026.08.17 23:52:47.884593 [ 433567 ] {} <Trace> DatabaseCatalog: Shutting down system databases | ||
| 34 | 2026.08.17 23:52:47.884597 [ 433567 ] {} <Trace> Context: Shutting down DDLWorker | ||
| 35 | 2026.08.17 23:52:47.884598 [ 433567 ] {} <Trace> Context: Shutting down caches | ||
| 36 | 2026.08.17 23:52:47.884621 [ 433567 ] {} <Trace> BackgroundSchedulePool/BgBufSchPool: Waiting for threads to finish. | ||
| 37 | 2026.08.17 23:52:47.884765 [ 433567 ] {} <Trace> BackgroundSchedulePool/BgSchPool: Waiting for threads to finish. | ||
| 38 | 2026.08.17 23:52:47.884807 [ 433567 ] {} <Trace> BackgroundSchedulePool/BgDistSchPool: Waiting for threads to finish. | ||
| 39 | 2026.08.17 23:52:47.884835 [ 433567 ] {} <Trace> BackgroundSchedulePool/BgMBSchPool: Waiting for threads to finish. | ||
| 40 | 2026.08.17 23:52:47.884984 [ 433567 ] {} <Error> Application: Code: 210. DB::Exception: Listen [::1]:19401 failed: Poco::Exception. Code: 1000, e.code() = 98, Net Exception: Address already in use: [::1]:19401 (version 23.8.7.24 (official build)). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): | ||
| 41 | |||
| 42 | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 43 | 1. DB::Exception::Exception<String const&, int&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int&>::type, std::type_identity<String>::type>, String const&, int&, String&&) @ 0x000000000c6cf2f8 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 44 | 2. DB::Server::createServer(Poco::Util::AbstractConfiguration&, String const&, char const*, bool, bool, std::vector<DB::ProtocolServerAdapter, std::allocator<DB::ProtocolServerAdapter>>&, std::function<DB::ProtocolServerAdapter (unsigned short)>&&) const @ 0x000000000c6cf057 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 45 | 3. DB::Server::createInterserverServers(Poco::Util::AbstractConfiguration&, std::vector<String, std::allocator<String>> const&, bool, Poco::ThreadPool&, DB::AsynchronousMetrics&, std::vector<DB::ProtocolServerAdapter, std::allocator<DB::ProtocolServerAdapter>>&, bool, DB::ServerType const&) @ 0x000000000c6eacb6 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 46 | 4. DB::Server::main(std::vector<String, std::allocator<String>> const&) @ 0x000000000c6df0c7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 47 | 5. Poco::Util::Application::run() @ 0x0000000015b6da26 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 48 | 6. DB::Server::run() @ 0x000000000c6cf89e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 49 | 7. Poco::Util::ServerApplication::run(int, char**) @ 0x0000000015b7cb39 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 50 | 8. mainEntryClickHouseServer(int, char**) @ 0x000000000c6cc66a in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 51 | 9. main @ 0x000000000715572f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 52 | 10. ? @ 0x00007f80c4fc0d90 in ? | ||
| 53 | 11. ? @ 0x00007f80c4fc0e40 in ? | ||
| 54 | 12. _start @ 0x000000000634fa2e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse | ||
| 55 | (version 23.8.7.24 (official build)) | ||
| 56 | 2026.08.17 23:52:47.885028 [ 433567 ] {} <Error> Application: DB::Exception: Listen [::1]:19401 failed: Poco::Exception. Code: 1000, e.code() = 98, Net Exception: Address already in use: [::1]:19401 (version 23.8.7.24 (official build)) | ||
| 57 | 2026.08.17 23:52:47.885031 [ 433567 ] {} <Information> Application: shutting down | ||
| 58 | 2026.08.17 23:52:47.885033 [ 433567 ] {} <Debug> Application: Uninitializing subsystem: Logging Subsystem | ||
| 59 | 2026.08.17 23:52:47.885219 [ 433568 ] {} <Trace> BaseDaemon: Received signal -2 | ||
| 60 | 2026.08.17 23:52:47.885234 [ 433568 ] {} <Information> BaseDaemon: Stop SignalListener thread | ||
| 61 | 2026.08.17 23:52:47.898459 [ 433557 ] {} <Information> Application: Child process exited normally with code 70. | ||