2026.08.17 23:52:47.705105 [ 433554 ] {} SentryWriter: Sending crash reports is disabled 2026.08.17 23:52:47.705139 [ 433554 ] {} Pipe: Pipe capacity is 1.00 MiB 2026.08.17 23:52:47.739354 [ 433554 ] {} : Starting ClickHouse Keeper 23.8.7.24(revision : 54477, git hash: 812b95e14ba8cf744bf1d70e6de607cf130a79fa, build id: 6484B75ECB16F4FF87A88E10249A45E1BDBDE2E5), PID 433554 2026.08.17 23:52:47.741539 [ 433554 ] {} Application: starting up 2026.08.17 23:52:47.741547 [ 433554 ] {} Application: OS Name = Linux, OS Version = 5.15.0-1015-aws, OS Architecture = x86_64 2026.08.17 23:52:47.745770 [ 433554 ] {} Application: Initializing DateLUT. 2026.08.17 23:52:47.745775 [ 433554 ] {} Application: Initialized DateLUT with time zone 'UTC'. 2026.08.17 23:52:47.746041 [ 433554 ] {} AsynchronousMetrics: Scanning /sys/class/thermal 2026.08.17 23:52:47.746048 [ 433554 ] {} AsynchronousMetrics: Scanning /sys/block 2026.08.17 23:52:47.746072 [ 433554 ] {} AsynchronousMetrics: Scanning /sys/devices/system/edac 2026.08.17 23:52:47.746079 [ 433554 ] {} AsynchronousMetrics: Scanning /sys/class/hwmon 2026.08.17 23:52:47.746172 [ 433554 ] {} Context: Cannot connect to ZooKeeper (or Keeper) before internal Keeper start, will wait for Keeper synchronously 2026.08.17 23:52:47.746177 [ 433554 ] {} KeeperDispatcher: Initializing storage dispatcher 2026.08.17 23:52:47.746317 [ 433554 ] {} KeeperContext: Keeper feature flag FILTERED_LIST: enabled 2026.08.17 23:52:47.746320 [ 433554 ] {} KeeperContext: Keeper feature flag MULTI_READ: enabled 2026.08.17 23:52:47.746322 [ 433554 ] {} KeeperContext: Keeper feature flag CHECK_NOT_EXISTS: disabled 2026.08.17 23:52:47.746399 [ 433554 ] {} KeeperSnapshotManager: Reading from disk LocalSnapshotDisk 2026.08.17 23:52:47.746459 [ 433554 ] {} KeeperLogStore: Reading from disk LocalLogDisk 2026.08.17 23:52:47.746467 [ 433554 ] {} KeeperLogStore: No logs exists in /var/tmp/omicron_tmp/integration_test-642120957f655e64-test_cluster.433551.1-oximeter-clickward-test/keeper-3/coordination/log. It's Ok if it's the first run of clickhouse-keeper. 2026.08.17 23:52:47.746535 [ 433554 ] {} KeeperLogStore: force_sync enabled 2026.08.17 23:52:47.746537 [ 433554 ] {} KeeperDispatcher: Waiting server to initialize 2026.08.17 23:52:47.746544 [ 433554 ] {} KeeperStateMachine: Totally have 0 snapshots 2026.08.17 23:52:47.746549 [ 433554 ] {} KeeperStateMachine: No existing snapshots, last committed log index 0 2026.08.17 23:52:47.746585 [ 433554 ] {} KeeperLogStore: Removing all changelogs 2026.08.17 23:52:47.746593 [ 433554 ] {} Changelog: Starting new changelog changelog_1_100000.bin.zstd 2026.08.17 23:52:47.746617 [ 433554 ] {} KeeperServer: Preprocessing 0 log entries 2026.08.17 23:52:47.746618 [ 433554 ] {} KeeperServer: Preprocessing done 2026.08.17 23:52:47.746622 [ 433554 ] {} KeeperServer: No config in log store and snapshot, probably it's initial run. Will use config from .xml on disk 2026.08.17 23:52:47.748066 [ 433554 ] {} RaftInstance: got exception: bind: Address already in use [system:98] on port 19103 2026.08.17 23:52:47.755955 [ 433554 ] {} void DB::KeeperDispatcher::initialize(const Poco::Util::AbstractConfiguration &, bool, bool, const MultiVersion::Version &): Code: 568. DB::Exception: Cannot create interserver listener on port 19103. (RAFT_ERROR), Stack trace (when copying this message, always include the lines below): 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 1. DB::Exception::Exception(int, FormatStringHelperImpl::type>, int&&) @ 0x000000000cf22a5c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 2. DB::KeeperServer::launchRaftServer(Poco::Util::AbstractConfiguration const&, bool) @ 0x00000000136faf3d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 3. DB::KeeperServer::startup(Poco::Util::AbstractConfiguration const&, bool) @ 0x00000000136fe39e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 4. DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::shared_ptr const&) @ 0x00000000136df8d8 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 5. DB::Context::initializeKeeperDispatcher(bool) const @ 0x000000001166ff82 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 6. DB::Keeper::main(std::vector> const&) @ 0x000000000c8a65da in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 7. Poco::Util::Application::run() @ 0x0000000015b6da26 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 8. DB::Keeper::run() @ 0x000000000c8a375e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 9. Poco::Util::ServerApplication::run(int, char**) @ 0x0000000015b7cb39 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 10. mainEntryClickHouseKeeper(int, char**) @ 0x000000000c8a2918 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 11. main @ 0x000000000715572f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 12. ? @ 0x00007f3de5b53d90 in ? 13. ? @ 0x00007f3de5b53e40 in ? 14. _start @ 0x000000000634fa2e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse (version 23.8.7.24 (official build)) 2026.08.17 23:52:47.756002 [ 433554 ] {} Context: Shutting down named sessions 2026.08.17 23:52:47.756006 [ 433554 ] {} Context: Shutting down database catalog 2026.08.17 23:52:47.756008 [ 433554 ] {} DatabaseCatalog: Shutting down system databases 2026.08.17 23:52:47.756011 [ 433554 ] {} Context: Shutting down DDLWorker 2026.08.17 23:52:47.756013 [ 433554 ] {} Context: Shutting down caches 2026.08.17 23:52:47.756042 [ 433554 ] {} KeeperDispatcher: Shutting down storage dispatcher 2026.08.17 23:52:47.756095 [ 433554 ] {} KeeperServer: RAFT doesn't start, shutdown not required 2026.08.17 23:52:47.758791 [ 433591 ] {} KeeperLogStore: Raft server is not set in LogStore. 2026.08.17 23:52:47.758817 [ 433554 ] {} KeeperSnapshotManagerS3: Shutting down KeeperSnapshotManagerS3 2026.08.17 23:52:47.758827 [ 433554 ] {} KeeperSnapshotManagerS3: KeeperSnapshotManagerS3 shut down 2026.08.17 23:52:47.758829 [ 433554 ] {} KeeperDispatcher: Dispatcher shut down 2026.08.17 23:52:47.758900 [ 433594 ] {} RaftInstance: end of asio worker thread, remaining threads: 15 2026.08.17 23:52:47.758961 [ 433597 ] {} RaftInstance: end of asio worker thread, remaining threads: 13 2026.08.17 23:52:47.758972 [ 433607 ] {} RaftInstance: end of asio worker thread, remaining threads: 9 2026.08.17 23:52:47.758971 [ 433599 ] {} RaftInstance: end of asio worker thread, remaining threads: 6 2026.08.17 23:52:47.758983 [ 433598 ] {} RaftInstance: end of asio worker thread, remaining threads: 4 2026.08.17 23:52:47.758986 [ 433605 ] {} RaftInstance: end of asio worker thread, remaining threads: 5 2026.08.17 23:52:47.759036 [ 433608 ] {} RaftInstance: end of asio worker thread, remaining threads: 7 2026.08.17 23:52:47.759063 [ 433609 ] {} RaftInstance: end of asio worker thread, remaining threads: 2 2026.08.17 23:52:47.759088 [ 433596 ] {} RaftInstance: end of asio worker thread, remaining threads: 0 2026.08.17 23:52:47.759117 [ 433604 ] {} RaftInstance: end of asio worker thread, remaining threads: 10 2026.08.17 23:52:47.759136 [ 433600 ] {} RaftInstance: end of asio worker thread, remaining threads: 8 2026.08.17 23:52:47.759136 [ 433595 ] {} RaftInstance: end of asio worker thread, remaining threads: 14 2026.08.17 23:52:47.759144 [ 433602 ] {} RaftInstance: end of asio worker thread, remaining threads: 3 2026.08.17 23:52:47.759150 [ 433601 ] {} RaftInstance: end of asio worker thread, remaining threads: 11 2026.08.17 23:52:47.759160 [ 433603 ] {} RaftInstance: end of asio worker thread, remaining threads: 12 2026.08.17 23:52:47.759180 [ 433606 ] {} RaftInstance: end of asio worker thread, remaining threads: 1 2026.08.17 23:52:47.759263 [ 433554 ] {} KeeperLogStore: Changelog is shut down 2026.08.17 23:52:47.759366 [ 433554 ] {} Application: Code: 568. DB::Exception: Cannot create interserver listener on port 19103. (RAFT_ERROR), Stack trace (when copying this message, always include the lines below): 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6487f7 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 1. DB::Exception::Exception(int, FormatStringHelperImpl::type>, int&&) @ 0x000000000cf22a5c in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 2. DB::KeeperServer::launchRaftServer(Poco::Util::AbstractConfiguration const&, bool) @ 0x00000000136faf3d in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 3. DB::KeeperServer::startup(Poco::Util::AbstractConfiguration const&, bool) @ 0x00000000136fe39e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 4. DB::KeeperDispatcher::initialize(Poco::Util::AbstractConfiguration const&, bool, bool, std::shared_ptr const&) @ 0x00000000136df8d8 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 5. DB::Context::initializeKeeperDispatcher(bool) const @ 0x000000001166ff82 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 6. DB::Keeper::main(std::vector> const&) @ 0x000000000c8a65da in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 7. Poco::Util::Application::run() @ 0x0000000015b6da26 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 8. DB::Keeper::run() @ 0x000000000c8a375e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 9. Poco::Util::ServerApplication::run(int, char**) @ 0x0000000015b7cb39 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 10. mainEntryClickHouseKeeper(int, char**) @ 0x000000000c8a2918 in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 11. main @ 0x000000000715572f in /work/oxidecomputer/omicron/out/clickhouse/clickhouse 12. ? @ 0x00007f3de5b53d90 in ? 13. ? @ 0x00007f3de5b53e40 in ? 14. _start @ 0x000000000634fa2e in /work/oxidecomputer/omicron/out/clickhouse/clickhouse (version 23.8.7.24 (official build)) 2026.08.17 23:52:47.759386 [ 433554 ] {} Application: DB::Exception: Cannot create interserver listener on port 19103 2026.08.17 23:52:47.759389 [ 433554 ] {} Application: shutting down 2026.08.17 23:52:47.759391 [ 433554 ] {} Application: Uninitializing subsystem: Logging Subsystem 2026.08.17 23:52:47.759412 [ 433562 ] {} BaseDaemon: Received signal -2 2026.08.17 23:52:47.759421 [ 433562 ] {} BaseDaemon: Stop SignalListener thread