Max level shown:
1 2024.09.28 15:28:37.133574 [ 43353 ] {} <Warning> KeeperLogStore: No logs exists in /var/tmp/omicron_tmp/integration_test-191fb91badeb8f2e-test_raft_config_parsing.43348.1-oximeter-clickward-test/keeper-2/coordination/log. It's Ok if it's the first run of clickhouse-keeper.
2 2024.09.28 15:28:37.133756 [ 43353 ] {} <Warning> KeeperLogStore: Removing all changelogs
3 2024.09.28 15:28:37.134288 [ 43353 ] {} <Warning> KeeperStateManager: No state was read
4 2024.09.28 15:28:38.306750 [ 43384 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
5 2024.09.28 15:28:38.347908 [ 43384 ] {} <Warning> RaftInstance: peer (3) response error: failed to read response to peer 3, ::1:39103, error 2, End of file
6 2024.09.28 15:28:38.372540 [ 43380 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
7 2024.09.28 15:28:38.375306 [ 43380 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
8 2024.09.28 15:28:38.406232 [ 43383 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
9 2024.09.28 15:28:38.424774 [ 43386 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
10 2024.09.28 15:28:38.549889 [ 43386 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
11 2024.09.28 15:28:38.627553 [ 43385 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
12 2024.09.28 15:28:38.721982 [ 43383 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
13 2024.09.28 15:28:38.890889 [ 43382 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
14 2024.09.28 15:28:39.208355 [ 43383 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
15 2024.09.28 15:28:39.720287 [ 43379 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
16 2024.09.28 15:28:40.321852 [ 43379 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
17 2024.09.28 15:28:40.853425 [ 43381 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
18 2024.09.28 15:28:41.354430 [ 43383 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
19 2024.09.28 15:28:41.873478 [ 43382 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
20 2024.09.28 15:28:42.386201 [ 43384 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
21 2024.09.28 15:28:42.897208 [ 43385 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
22 2024.09.28 15:28:43.398121 [ 43385 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
23 2024.09.28 15:28:43.924759 [ 43380 ] {} <Warning> RaftInstance: peer (3) response error: failed to connect to peer 3, ::1:39103, error 111, Connection refused
24 2024.09.28 15:28:44.502441 [ 43381 ] {} <Warning> RaftInstance: too verbose RPC error on peer (3), will suppress it from now
25 2024.09.28 15:39:03.275050 [ 43386 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 751 ms ago
26 2024.09.28 15:43:38.959163 [ 43386 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 751 ms ago
27 2024.09.28 15:43:39.709277 [ 43381 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 1501 ms ago
28 2024.09.28 15:43:40.459420 [ 43379 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 2251 ms ago
29 2024.09.28 15:43:41.210864 [ 43381 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 3002 ms ago
30 2024.09.28 15:43:41.960987 [ 43380 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 3752 ms ago
31 2024.09.28 15:43:42.711102 [ 43384 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 4503 ms ago
32 2024.09.28 15:43:42.909198 [ 43388 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 4701 ms ago
33 2024.09.28 15:43:42.911244 [ 43384 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 4703 ms ago
34 2024.09.28 15:43:43.461222 [ 43383 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 5253 ms ago
35 2024.09.28 15:43:44.211365 [ 43383 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 6003 ms ago
36 2024.09.28 15:43:44.961502 [ 43381 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 6753 ms ago
37 2024.09.28 15:43:45.711622 [ 43384 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 7503 ms ago
38 2024.09.28 15:43:46.461741 [ 43382 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 8253 ms ago
39 2024.09.28 15:43:47.211878 [ 43381 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 9003 ms ago
40 2024.09.28 15:43:47.962006 [ 43386 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 9753 ms ago
41 2024.09.28 15:43:48.712129 [ 43380 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 10504 ms ago
42 2024.09.28 15:43:49.462222 [ 43383 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 11254 ms ago
43 2024.09.28 15:43:50.212329 [ 43386 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 12004 ms ago
44 2024.09.28 15:43:50.962453 [ 43379 ] {} <Warning> RaftInstance: skipped sending msg to 3 too long time, last msg sent 12754 ms ago
45 2024.09.28 15:43:51.712573 [ 43381 ] {} <Warning> RaftInstance: long pause warning to 3 is too verbose, will suppress it from now
46 2024.09.28 15:44:02.912883 [ 43382 ] {} <Warning> RaftInstance: recovered from long pause to peer 3, 24 warnings, 3 ms, 1 times