2024.10.22 00:06:59.936072 [ 468 ] {} <Warning> Context: Server was built in debug mode. It will work slowly.
2024.10.22 00:06:59.939093 [ 468 ] {} <Warning> Context: ThreadFuzzer is enabled. Application will run slowly and unstable.
2024.10.22 00:07:00.034053 [ 468 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.10.22 00:07:18.760904 [ 468 ] {} <Warning> Context: The setting 'merge_tree_metadata_cache' is enabled. But the feature of 'metadata cache in RocksDB' is experimental and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. See the corresponding report at https://github.com/ClickHouse/ClickHouse/issues/51182
2024.10.22 00:07:36.912321 [ 468 ] {} <Warning> KeeperLogStore: No logs exists in /var/lib/clickhouse/coordination/logs. It's Ok if it's the first run of clickhouse-keeper.
2024.10.22 00:07:36.923808 [ 468 ] {} <Warning> KeeperLogStore: Removing all changelogs
2024.10.22 00:07:36.951133 [ 468 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.10.22 00:07:37.069622 [ 468 ] {} <Warning> KeeperStateManager: No state was read
2024.10.22 00:07:37.072637 [ 468 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.10.22 00:07:37.074374 [ 468 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.10.22 00:07:37.178326 [ 794 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.10.22 00:07:40.049749 [ 468 ] {} <Warning> Access(local_directory): File /var/lib/clickhouse/access/users.list doesn't exist
2024.10.22 00:07:40.051188 [ 468 ] {} <Warning> Access(local_directory): Recovering lists in directory /var/lib/clickhouse/access/
2024.10.22 00:07:46.104672 [ 468 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.10.22 00:12:27.930271 [ 969 ] {} <Error> ZooKeeperClient: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:26: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000013cf152c in /usr/bin/clickhouse
6. /build/src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000013cf07a0 in /usr/bin/clickhouse
7. /build/src/IO/ReadBuffer.h:71: DB::ReadBuffer::next() @ 0x000000000b21bd7c in /usr/bin/clickhouse
8. /build/src/IO/ReadBuffer.h:105: DB::ReadBuffer::read(char*, unsigned long) @ 0x000000000b21b973 in /usr/bin/clickhouse
9. /build/src/IO/ReadBuffer.h:196: DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b5e4468 in /usr/bin/clickhouse
10. /build/src/IO/ReadHelpers.h:117: void DB::readPODBinary<int>(int&, DB::ReadBuffer&) @ 0x0000000013999f02 in /usr/bin/clickhouse
11. /build/src/IO/ReadHelpers.h:1099: void DB::readBinary<int>(int&, DB::ReadBuffer&) @ 0x0000000013999ebd in /usr/bin/clickhouse
12. /build/src/IO/ReadHelpers.h:1136: void DB::readBinaryEndian<(std::endian)64206, int>(int&, DB::ReadBuffer&) @ 0x000000001e60277d in /usr/bin/clickhouse
13. /build/src/IO/ReadHelpers.h:1149: void DB::readBinaryBigEndian<int>(int&, DB::ReadBuffer&) @ 0x000000001e60273d in /usr/bin/clickhouse
14. /build/src/Common/ZooKeeper/ZooKeeperIO.h:51: void Coordination::read<int>(int&, DB::ReadBuffer&) @ 0x000000001e5fdbbd in /usr/bin/clickhouse
15. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:284: void Coordination::ZooKeeper::read<int>(int&) @ 0x000000001f30be14 in /usr/bin/clickhouse
16. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:744: Coordination::ZooKeeper::receiveEvent() @ 0x000000001f30124c in /usr/bin/clickhouse
17. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:710: Coordination::ZooKeeper::receiveThread() @ 0x000000001f300cc4 in /usr/bin/clickhouse
18. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:356: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1::operator()() const @ 0x000000001f306af8 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&>()()) std::__invoke[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&) @ 0x000000001f306ad5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001f306ab9 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&) @ 0x000000001f306a1d in /usr/bin/clickhouse
22. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()::operator()() @ 0x000000001f306945 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&) @ 0x000000001f3068b5 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&) @ 0x000000001f306895 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001f306875 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001f306840 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
29. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x0000000013a5c843 in /usr/bin/clickhouse
30. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 00:12:29.171221 [ 968 ] {} <Error> ZooKeeperClient: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:37914 -> [::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:26: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x0000000013cf30a8 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:107: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000013cf26c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:48: DB::WriteBuffer::next() @ 0x000000000b207e8d in /usr/bin/clickhouse
8. /build/src/Common/ZooKeeper/ZooKeeperCommon.cpp:51: Coordination::ZooKeeperRequest::write(DB::WriteBuffer&) const @ 0x000000001f2cf9e5 in /usr/bin/clickhouse
9. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:645: Coordination::ZooKeeper::sendThread() @ 0x000000001f2fffbc in /usr/bin/clickhouse
10. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:355: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0::operator()() const @ 0x000000001f3062b8 in /usr/bin/clickhouse
11. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&>()()) std::__invoke[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&) @ 0x000000001f306295 in /usr/bin/clickhouse
12. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001f306279 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&, std::tuple<>&) @ 0x000000001f3061dd in /usr/bin/clickhouse
14. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'()::operator()() @ 0x000000001f306105 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'()&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&) @ 0x000000001f306075 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'()&) @ 0x000000001f306055 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001f306035 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001f306000 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x0000000013a5c843 in /usr/bin/clickhouse
22. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x0000000013a649bd in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x0000000013a644a2 in /usr/bin/clickhouse
26. ? @ 0x00007fb76fa68ac3 in ?
27. ? @ 0x00007fb76fafa850 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 00:12:30.752344 [ 969 ] {} <Error> ZooKeeperClient: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/base/poco/Foundation/src/Exception.cpp:145: Poco::RuntimeException::RuntimeException(String const&, int) @ 0x000000002495e5e7 in /usr/bin/clickhouse
4. /build/base/poco/Foundation/src/Exception.cpp:164: Poco::IOException::IOException(String const&, int) @ 0x0000000024962727 in /usr/bin/clickhouse
5. /build/base/poco/Net/src/NetException.cpp:26: Poco::Net::NetException::NetException(String const&, int) @ 0x0000000024769587 in /usr/bin/clickhouse
6. /build/base/poco/Net/src/SocketImpl.cpp:995: Poco::Net::SocketImpl::error(int, String const&) @ 0x0000000024778ecb in /usr/bin/clickhouse
7. /build/base/poco/Net/src/SocketImpl.cpp:914: Poco::Net::SocketImpl::error() @ 0x0000000024777869 in /usr/bin/clickhouse
8. /build/base/poco/Net/src/SocketImpl.cpp:272: Poco::Net::SocketImpl::shutdown() @ 0x000000002477a292 in /usr/bin/clickhouse
9. /build/base/poco/Net/src/StreamSocket.cpp:109: Poco::Net::StreamSocket::shutdown() @ 0x000000002477dfbe in /usr/bin/clickhouse
10. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:961: Coordination::ZooKeeper::finalize(bool, bool, String const&) @ 0x000000001f2f963a in /usr/bin/clickhouse
11. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:732: Coordination::ZooKeeper::receiveThread() @ 0x000000001f3010c3 in /usr/bin/clickhouse
12. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:356: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1::operator()() const @ 0x000000001f306af8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&>()()) std::__invoke[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&) @ 0x000000001f306ad5 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001f306ab9 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&) @ 0x000000001f306a1d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()::operator()() @ 0x000000001f306945 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&) @ 0x000000001f3068b5 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&) @ 0x000000001f306895 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001f306875 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001f306840 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x0000000013a5c843 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x0000000013a649bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x0000000013a644a2 in /usr/bin/clickhouse
28. ? @ 0x00007fb76fa68ac3 in ?
29. ? @ 0x00007fb76fafa850 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 00:14:07.843103 [ 786 ] {} <Error> RaftInstance: failed to accept a rpc connection due to error 125, Operation canceled
2024.10.22 00:14:08.270497 [ 785 ] {} <Warning> KeeperLogStore: Raft server is not set in LogStore.
2024.10.22 00:14:10.768660 [ 468 ] {} <Warning> KeeperLogStore: Changelog is shut down
2024.10.22 00:14:18.628457 [ 1034 ] {} <Warning> Context: Server was built in debug mode. It will work slowly.
2024.10.22 00:14:18.642214 [ 1034 ] {} <Warning> Context: ThreadFuzzer is enabled. Application will run slowly and unstable.
2024.10.22 00:14:18.772418 [ 1034 ] {} <Warning> Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production.
2024.10.22 00:14:36.804208 [ 1034 ] {} <Warning> Context: The setting 'merge_tree_metadata_cache' is enabled. But the feature of 'metadata cache in RocksDB' is experimental and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. See the corresponding report at https://github.com/ClickHouse/ClickHouse/issues/51182
2024.10.22 00:14:53.795364 [ 1034 ] {} <Warning> KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647.
2024.10.22 00:14:53.836600 [ 1034 ] {} <Warning> RaftInstance: invalid election timeout lower bound detected, adjusted to 0
2024.10.22 00:14:53.842137 [ 1034 ] {} <Warning> RaftInstance: invalid election timeout upper bound detected, adjusted to 0
2024.10.22 00:14:53.945042 [ 1350 ] {} <Warning> RaftInstance: Election timeout, initiate leader election
2024.10.22 00:15:15.836460 [ 1034 ] {} <Warning> Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring.
2024.10.22 00:20:02.339836 [ 1447 ] {} <Error> ZooKeeperClient: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:26: DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000013cf152c in /usr/bin/clickhouse
6. /build/src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000013cf07a0 in /usr/bin/clickhouse
7. /build/src/IO/ReadBuffer.h:71: DB::ReadBuffer::next() @ 0x000000000b21bd7c in /usr/bin/clickhouse
8. /build/src/IO/ReadBuffer.h:105: DB::ReadBuffer::read(char*, unsigned long) @ 0x000000000b21b973 in /usr/bin/clickhouse
9. /build/src/IO/ReadBuffer.h:196: DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b5e4468 in /usr/bin/clickhouse
10. /build/src/IO/ReadHelpers.h:117: void DB::readPODBinary<int>(int&, DB::ReadBuffer&) @ 0x0000000013999f02 in /usr/bin/clickhouse
11. /build/src/IO/ReadHelpers.h:1099: void DB::readBinary<int>(int&, DB::ReadBuffer&) @ 0x0000000013999ebd in /usr/bin/clickhouse
12. /build/src/IO/ReadHelpers.h:1136: void DB::readBinaryEndian<(std::endian)64206, int>(int&, DB::ReadBuffer&) @ 0x000000001e60277d in /usr/bin/clickhouse
13. /build/src/IO/ReadHelpers.h:1149: void DB::readBinaryBigEndian<int>(int&, DB::ReadBuffer&) @ 0x000000001e60273d in /usr/bin/clickhouse
14. /build/src/Common/ZooKeeper/ZooKeeperIO.h:51: void Coordination::read<int>(int&, DB::ReadBuffer&) @ 0x000000001e5fdbbd in /usr/bin/clickhouse
15. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:284: void Coordination::ZooKeeper::read<int>(int&) @ 0x000000001f30be14 in /usr/bin/clickhouse
16. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:744: Coordination::ZooKeeper::receiveEvent() @ 0x000000001f30124c in /usr/bin/clickhouse
17. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:710: Coordination::ZooKeeper::receiveThread() @ 0x000000001f300cc4 in /usr/bin/clickhouse
18. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:356: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1::operator()() const @ 0x000000001f306af8 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&>()()) std::__invoke[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&) @ 0x000000001f306ad5 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001f306ab9 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&) @ 0x000000001f306a1d in /usr/bin/clickhouse
22. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()::operator()() @ 0x000000001f306945 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&) @ 0x000000001f3068b5 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&) @ 0x000000001f306895 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001f306875 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001f306840 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
29. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x0000000013a5c843 in /usr/bin/clickhouse
30. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 00:20:03.621013 [ 1505 ] {} <Error> ZooKeeperClient: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:41272 -> [::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/NetException.h:26: DB::NetException::NetException<String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&) @ 0x0000000013cf30a8 in /usr/bin/clickhouse
6. /build/src/IO/WriteBufferFromPocoSocket.cpp:107: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000013cf26c4 in /usr/bin/clickhouse
7. /build/src/IO/WriteBuffer.h:48: DB::WriteBuffer::next() @ 0x000000000b207e8d in /usr/bin/clickhouse
8. /build/src/Common/ZooKeeper/ZooKeeperCommon.cpp:51: Coordination::ZooKeeperRequest::write(DB::WriteBuffer&) const @ 0x000000001f2cf9e5 in /usr/bin/clickhouse
9. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:645: Coordination::ZooKeeper::sendThread() @ 0x000000001f2fffbc in /usr/bin/clickhouse
10. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:355: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0::operator()() const @ 0x000000001f3062b8 in /usr/bin/clickhouse
11. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&>()()) std::__invoke[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&) @ 0x000000001f306295 in /usr/bin/clickhouse
12. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001f306279 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&, std::tuple<>&) @ 0x000000001f3061dd in /usr/bin/clickhouse
14. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'()::operator()() @ 0x000000001f306105 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'()&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&) @ 0x000000001f306075 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'()&) @ 0x000000001f306055 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001f306035 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001f306000 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x0000000013a5c843 in /usr/bin/clickhouse
22. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x0000000013a649bd in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x0000000013a644a2 in /usr/bin/clickhouse
26. ? @ 0x00007fc9b4da9ac3 in ?
27. ? @ 0x00007fc9b4e3b850 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 00:20:06.514355 [ 1447 ] {} <Error> ZooKeeperClient: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/base/poco/Foundation/src/Exception.cpp:145: Poco::RuntimeException::RuntimeException(String const&, int) @ 0x000000002495e5e7 in /usr/bin/clickhouse
4. /build/base/poco/Foundation/src/Exception.cpp:164: Poco::IOException::IOException(String const&, int) @ 0x0000000024962727 in /usr/bin/clickhouse
5. /build/base/poco/Net/src/NetException.cpp:26: Poco::Net::NetException::NetException(String const&, int) @ 0x0000000024769587 in /usr/bin/clickhouse
6. /build/base/poco/Net/src/SocketImpl.cpp:995: Poco::Net::SocketImpl::error(int, String const&) @ 0x0000000024778ecb in /usr/bin/clickhouse
7. /build/base/poco/Net/src/SocketImpl.cpp:914: Poco::Net::SocketImpl::error() @ 0x0000000024777869 in /usr/bin/clickhouse
8. /build/base/poco/Net/src/SocketImpl.cpp:272: Poco::Net::SocketImpl::shutdown() @ 0x000000002477a292 in /usr/bin/clickhouse
9. /build/base/poco/Net/src/StreamSocket.cpp:109: Poco::Net::StreamSocket::shutdown() @ 0x000000002477dfbe in /usr/bin/clickhouse
10. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:961: Coordination::ZooKeeper::finalize(bool, bool, String const&) @ 0x000000001f2f963a in /usr/bin/clickhouse
11. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:732: Coordination::ZooKeeper::receiveThread() @ 0x000000001f3010c3 in /usr/bin/clickhouse
12. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:356: Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1::operator()() const @ 0x000000001f306af8 in /usr/bin/clickhouse
13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&>()()) std::__invoke[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&) @ 0x000000001f306ad5 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001f306ab9 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&, std::tuple<>&) @ 0x000000001f306a1d in /usr/bin/clickhouse
16. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()::operator()() @ 0x000000001f306945 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&) @ 0x000000001f3068b5 in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'()&) @ 0x000000001f306895 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001f306875 in /usr/bin/clickhouse
20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<true>::ThreadFromGlobalPoolImpl<Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector<Coordination::ZooKeeper::Node, std::allocator<Coordination::ZooKeeper::Node>> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr<DB::ZooKeeperLog>, std::optional<std::function<void (unsigned long, Coordination::ZooKeeper::Node const&)>>&&)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001f306840 in /usr/bin/clickhouse
21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
23. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x0000000013a5c843 in /usr/bin/clickhouse
24. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse
25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(std::tuple<void, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x0000000013a649bd in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x0000000013a644a2 in /usr/bin/clickhouse
28. ? @ 0x00007fc9b4da9ac3 in ?
29. ? @ 0x00007fc9b4e3b850 in ?
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 00:20:17.489018 [ 1389 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 00:24:32.527992 [ 1391 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 00:41:54.919999 [ 1398 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 00:57:16.332183 [ 1397 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:01:15.294587 [ 1387 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:03:15.306639 [ 1385 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:05:03.861722 [ 1391 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:10:39.030024 [ 1390 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:12:30.838450 [ 1396 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:35:39.368071 [ 1392 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:36:20.524853 [ 1385 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:39:48.408917 [ 1399 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:49:15.277117 [ 1395 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 01:57:02.911816 [ 1395 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:03:55.059736 [ 1397 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:06:20.734141 [ 1397 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:15:19.596198 [ 1387 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:23:02.315379 [ 1387 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:23:43.521699 [ 1399 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:23:55.154270 [ 1393 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:27:37.271217 [ 1390 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:28:49.327442 [ 1392 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:29:23.271261 [ 1398 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))
2024.10.22 02:33:37.678346 [ 1386 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse
1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse
2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse
3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse
4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse
5. /build/src/Common/Exception.h:101: DB::Exception::Exception<char const*, char const*>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse
6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse
7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse
8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse
9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse
10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse
11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::routine(std::shared_ptr<DB::TaskRuntimeData>) @ 0x000000001de56fed in /usr/bin/clickhouse
12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse
13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse
14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>()()) std::__invoke[abi:v15000]<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse
15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse
16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse
17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, StrongTypedef<unsigned long, CurrentMetrics::MetricTag>, std::basic_string_view<char, std::char_traits<char>>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse
18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse
21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse
22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse
23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse
24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse
25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse
26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval<void>()()) std::__invoke[abi:v15000]<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse
27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper<void, true>::__call<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse
28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse
29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse
30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func<void ()>::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse
31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function<void ()>::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse
 (version 23.8.16.42.altinitystable (altinity build))